Real-time tooth numbering method and system
By capturing tooth images on a mobile device and using the YOLOv8 algorithm to generate tooth bounding boxes and coordinate information, and combining vertical and horizontal distributions to generate a tooth number queue, the problem of tooth numbering in online consultations is solved, and efficient tooth numbering on mobile devices is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-09
- Publication Date
- 2026-04-24
AI Technical Summary
Current technology cannot effectively number patients' teeth during online consultations, nor can it use user-uploaded dental images for tooth numbering, which affects online communication and treatment.
The system captures images of teeth using a mobile device, uses the YOLOv8 algorithm to detect tooth bounding boxes and generate coordinate information, combines vertical and horizontal coordinate information to generate a tooth distribution queue, and numbers teeth based on feature lines and reference coordinates, while also considering tooth loss for adaptive numbering.
It enables the numbering of dental images on mobile devices without the need for specialized equipment, improving the applicability and accuracy of dental numbering and facilitating online and AI-based consultations.
Smart Images
Figure CN121921482A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the technical field of tooth recognition, and in particular to a real-time tooth numbering method, system, and storage medium. Background Technology
[0002] Dental health is a major concern, with more and more people experiencing or approaching certain dental health problems. During patient consultations and treatments, it is necessary to assign corresponding numbers to the patient's teeth. Subsequently, doctors use these numbers to represent the corresponding teeth when writing medical records and conducting treatments.
[0003] Currently, tooth numbering techniques rely on CT or CBCT images of the patient's teeth as the base image, or on 3D models constructed from collected dental information as the base object. However, these methods require specialized imaging equipment for acquiring the base images or models, and also necessitate the assistance and guidance of professional physicians.
[0004] With the continuous development of technology, more and more users are choosing to conduct preliminary dental health inquiries through online consultations and AI consultations before visiting hospitals in person. In this consultation model, patients only upload pictures of their teeth. In this case, the patient's tooth number cannot be obtained through relevant technologies. At the same time, the online doctor or intelligent AI cannot obtain the patient's tooth number information based on the uploaded dental pictures, which has a certain impact on subsequent online communication and online consultations. Summary of the Invention
[0005] To address the aforementioned technical problems, this application provides a real-time tooth numbering method, system, and storage medium.
[0006] Firstly, the real-time tooth numbering method provided in this application adopts the following technical solution: A real-time tooth numbering method, implemented based on images captured by a mobile device, includes the following steps: Obtain images of teeth; The tooth image is processed to generate a tooth frame corresponding to each tooth and tooth coordinate information corresponding to several tooth frames; Based on the ordinate of each tooth coordinate, longitudinal distribution information of the teeth is generated, and the longitudinal distribution information of the teeth is used to characterize whether the tooth frame is located on the upper side or the lower side of the tooth. Select the feature abscissa of the tooth frame on the same side and obtain the selected feature line based on the feature abscissa, wherein the same side is characterized as both being on the upper side or both being on the lower side; Based on the selected feature line, the tooth positioning reference position and its corresponding reference coordinate information are obtained. The tooth positioning reference position is characterized as a reference object used to determine whether the tooth is located on the left or right side. A left-right distribution queue of teeth is generated based on the tooth coordinate information corresponding to several tooth frames and the reference coordinate information; The tooth frames are numbered based on the left-right distribution queue of the teeth and the longitudinal distribution information of the teeth.
[0007] In other embodiments, generating longitudinal distribution information of teeth based on the ordinate in the tooth coordinate information corresponding to each of the tooth frames specifically includes: Traverse the tooth coordinate information of the tooth frame to obtain the minimum and maximum ordinates; Calculate the median of the minimum and maximum ordinates, and define it as the longitudinal median; Determine whether the ordinate of the upper left corner of each tooth frame is greater than the median of the vertical axis; If it is not greater than, then the tooth frame is determined to correspond to the lower side of the tooth; if it is greater than, then the vertical difference between the vertical coordinate corresponding to the upper left corner of the tooth frame and the vertical median is calculated, and it is determined whether the vertical difference is greater than a preset value. If the value is greater than the value, the tooth frame is determined to correspond to the upper side of the tooth; if the value is less than the value, the tooth frame corresponds to the lower side of the tooth.
[0008] In other embodiments, before obtaining the feature abscissas of the tooth frames on the same side based on a plurality of tooth frames and obtaining the selected feature line based on the feature abscissas, the following steps are further included: Obtain user's missing tooth registration information, and determine whether the user has missing teeth based on the user's missing tooth registration information; If the user does not have missing teeth, the first selected method is used to obtain the feature x-coordinate and the selected feature line is obtained based on the feature x-coordinate; If the user has missing teeth, the second selected method is used to obtain the feature x-coordinate and the selected feature line is obtained based on the feature x-coordinate.
[0009] In other embodiments, if the user does not have missing teeth, the first selection method is used to obtain the feature abscissa and the selected feature line is obtained based on the feature abscissa, including the following steps: If the user does not have any missing teeth, then all tooth frames located above the teeth are traversed to obtain the maximum and minimum horizontal coordinates, and the maximum and minimum horizontal coordinates are used as feature horizontal coordinates; Generate a selected feature line with the two described feature x-coordinates as endpoint x-coordinates and parallel to the x-axis.
[0010] In other embodiments, obtaining the tooth positioning reference position and its corresponding reference coordinate information based on the selected feature line includes the following steps: Obtain the midpoint of the selected feature line and use this midpoint as the tooth positioning reference. Also obtain the coordinate information corresponding to this midpoint as reference coordinate information.
[0011] In other embodiments, if the user has missing teeth, the second selection method is used to obtain the feature abscissa and the selected feature line is obtained based on the feature abscissa, including the following steps: If the user has missing teeth, the median abscissa of several tooth frames located on the upper side of the teeth is obtained and defined as the feature abscissa. The median abscissa represents the median of the abscissas corresponding to the two sides of each tooth frame. Along the horizontal axis, each of the tooth frames is selected as the target tooth frame, and a selected feature line is generated with the feature horizontal coordinates corresponding to the target tooth frame and the feature horizontal coordinates corresponding to all other tooth frames located on the fixed horizontal axis side as the endpoint horizontal coordinates and parallel to the horizontal axis.
[0012] In other embodiments, obtaining the tooth positioning reference position and its corresponding reference coordinate information based on the selected feature line includes the following steps: Obtain the midpoint of each of the selected feature lines, and obtain the minimum discrete point group based on several midpoints. The minimum discrete point group is characterized as the set of midpoints with the smallest degree of dispersion of the horizontal coordinate within a preset range. The abscissas of all midpoints within the minimum discrete point group are obtained and their mean values are calculated to generate the tooth positioning reference position and the corresponding reference coordinate information.
[0013] In other embodiments, generating a left-right distribution queue of teeth based on tooth coordinate information corresponding to several tooth frames and the reference coordinate information includes the following steps: Determine whether the abscissa corresponding to the reference coordinate information is located between the abscissas of the upper left and upper right corners of the tooth frame; If it is located, then determine the lateral distance between the tooth positioning reference point and the upper left and upper right corner points of the tooth frame; If the lateral distance between the upper left corner and the tooth positioning reference is greater than the lateral distance between the upper right corner and the tooth positioning reference, then the coordinate frame is located on the left side of the tooth; if the lateral distance between the upper left corner and the tooth positioning reference is less than the lateral distance between the upper right corner and the tooth positioning reference, then the coordinate frame is located on the right side of the tooth. Obtain the upper left corner of each tooth frame located on the left side of the tooth, and generate a left queue sorting based on the difference in horizontal coordinate between each upper left corner and the tooth positioning reference position; obtain the upper right corner of each tooth frame located on the right side of the tooth, and generate a right queue sorting based on the difference in horizontal coordinate between each upper right corner and the tooth positioning reference position.
[0014] In other embodiments, numbering a plurality of the tooth frames based on the left-right distribution queue and the longitudinal distribution information of the teeth further includes the step of: If a user has missing teeth, the missing tooth number is obtained based on the user's missing tooth registration information; Find the tooth number in the tooth image that is the same as the missing tooth number, and add one to the number of that tooth number and the numbers of all tooth frames with a higher number.
[0015] Secondly, the real-time tooth numbering system provided in this application adopts the following technical solution: A real-time tooth numbering system, based on images captured by a mobile device, is used to implement the aforementioned real-time tooth numbering method.
[0016] In summary, this application includes at least one of the following beneficial technical effects: 1. It can process, calculate, classify, and number dental images taken by users via mobile devices without the need for professional dental imaging equipment or 3D modeling. Users can directly generate corresponding number information for each dental image by uploading it, which is convenient for online consultations and AI consultations. 2. Different numbering methods can be selected based on whether the user has missing teeth, thereby improving the applicability and accuracy of tooth numbering. Attached Figure Description
[0017] Figure 1 This is a schematic diagram of the overall process of the tooth numbering method in this application; Figure 2 This is a schematic diagram of the tooth frame in the obtained tooth image in this application; Figure 3 This is a schematic diagram illustrating the acquisition of tooth positioning reference positions when there are no missing teeth in this application; Figure 4 This is a schematic diagram illustrating the acquisition of tooth positioning reference positions when there are missing teeth in this application; Figure 5 This is a schematic diagram of the tooth frames in the tooth image after numbering the teeth in this application; Figure 6 This is a schematic diagram illustrating the updating of tooth numbers when there are missing teeth in this application. Detailed Implementation
[0018] The following is in conjunction with the appendix Figure 1 - Appendix Figure 5 This application will be described in further detail below.
[0019] like Figure 1 As shown, a real-time tooth numbering method is implemented based on images captured by a mobile device, including but not limited to mobile phones, tablets, laptops, cameras, camcorders, action cameras, etc. Furthermore, in this application, the images captured by the mobile device are defined as frontal views including the user's teeth. After capturing the tooth image, the user manually or automatically uploads the image for tooth numbering.
[0020] When taking images of their teeth, users can use mouth openers, their hands, or others to ensure that all their teeth are included in the image, thus improving the accuracy of tooth identification.
[0021] The real-time tooth numbering method includes the following steps: S100, acquire tooth image.
[0022] Get images of teeth that are uploaded manually or automatically by the user.
[0023] S200: Perform image processing on the tooth image to generate a tooth frame corresponding to each tooth and tooth coordinate information corresponding to several tooth frames.
[0024] like Figure 2 As shown, the image processing of the teeth image is implemented based on YOLOv8. YOLO (You Only Look Once) is an algorithm for object detection that uses convolutional neural networks. Compared with traditional recognition algorithms, YOLO can not only predict class labels but also detect the location of objects. Therefore, it can not only classify an image into a single category but also detect multiple objects in an image. It applies a single neural network to the entire image, which divides the image into regions and predicts bounding boxes and probabilities for each object region.
[0025] YOLOv8 is the latest object detection model released to date, and it includes five models for detection, segmentation, and classification.
[0026] Based on the YOLVv8 algorithm, each object in the tooth image, i.e., the tooth itself, is detected, and a bounding box for each tooth is generated. The four sides of the bounding box correspond to the endpoints of the tooth's top, left, right, and bottom, respectively. Simultaneously, after generating the bounding boxes, corresponding tooth coordinate information is generated for each tooth. This coordinate information includes the overall coordinates of the bounding box and the coordinates of each corner point. It's important to note that the overall coordinates of a bounding box can be represented by the coordinates of two corner points; for example, the overall coordinates of tooth bounding box 'a' are (xa top left, ya top left) (xa bottom right, ya bottom right).
[0027] When acquiring tooth coordinate information, the x-axis and y-axis of the coordinate system are along the long and short sides of the image area in the tooth image. The image area represents the image display area of the tooth image, and the origin of the coordinate system can be selected at any corner point in the tooth image. In this embodiment, the origin of the coordinate system is located at the upper left corner point.
[0028] S300 generates longitudinal distribution information of teeth based on the ordinate in the coordinate information of each tooth.
[0029] The longitudinal distribution information of teeth is used to characterize whether the tooth frame is located on the upper or lower side of the tooth.
[0030] The location of each tooth frame is determined by its vertical coordinate. If the tooth frame is on the upper side of the tooth, the tooth within the frame is an upper tooth; if the tooth frame is on the lower side of the tooth, the tooth within the frame is a lower tooth.
[0031] S400: Select the feature x-coordinate of the tooth frame on the same side and obtain the selected feature line based on the feature x-coordinate.
[0032] Same side is characterized as both being on the upper side or both being on the lower side.
[0033] Obtain the characteristic abscissas corresponding to all tooth frames located on the upper side. The characteristic abscissas represent the abscissa values corresponding to points or lines with characteristic meanings obtained on the selected tooth frames. Selected feature lines are then obtained from several characteristic abscissas. These selected feature lines are used as reference positions for subsequent determination of the left and right positions of each tooth frame.
[0034] S500 obtains the tooth positioning reference position and its corresponding reference coordinate information based on the selected feature line.
[0035] The tooth positioning reference position is a reference object used to determine whether a tooth is located on the left or right.
[0036] Simultaneously, after identifying the tooth positioning reference point, the corresponding reference coordinate information is also obtained. This reference coordinate information includes the horizontal coordinate and may also include other coordinate information such as the vertical coordinate. S600 generates a left-right distribution queue of teeth based on the tooth coordinate information and reference coordinate information corresponding to several tooth frames.
[0037] By comparing the coordinates of several tooth frames with reference coordinates, it can be determined whether a tooth is located on the left or right side. Furthermore, based on the distance relationships between the coordinates of these tooth frames and the reference coordinates, the order of the tooth frames can be determined. This order and left / right position result in a left-right tooth distribution queue. This queue can be further divided into an upper left-right tooth distribution queue and a lower left-right tooth distribution queue.
[0038] S700, based on the left-right distribution queue and the longitudinal distribution information of teeth, numbers several tooth frames.
[0039] After obtaining the left-right distribution queue and the longitudinal distribution information of the teeth, we can know the upper and lower, left and right positions of each tooth frame, as well as the sorting relationship between each tooth frame. Combining the position and sorting relationship, we can number each tooth frame. After numbering, the number is written into the tooth frame for display, and the number corresponding to the tooth frame is the number corresponding to the tooth in that tooth frame.
[0040] Using the above method, after a user uploads a tooth image using a mobile device such as a smartphone, the image processing is used to obtain the tooth frame and its corresponding coordinates. The coordinates are then used to directly determine the vertical position of the teeth. A tooth positioning reference position is found that can be used to distinguish and compare the left and right positions of the teeth and to confirm the left and right distribution queue of the teeth. Finally, the vertical distribution information of the teeth corresponding to the entire tooth frame and the left and right distribution queue of the teeth determine the tooth frame and the number of the teeth within the tooth frame.
[0041] The numbering methods are all based on the processing and coordinate calculation of the user's two-dimensional image. They do not require professional dental imaging equipment or three-dimensional modeling. Users can directly generate the corresponding number information for the tooth image by uploading a tooth image, which is convenient for use in online consultations and AI consultations.
[0042] In other embodiments, the longitudinal distribution information of teeth is generated based on the ordinate in the tooth coordinate information corresponding to each tooth frame, including the following steps: S310, traverse the tooth coordinate information of the tooth frame to obtain the minimum and maximum ordinates.
[0043] Each coordinate frame has several different ordinate values. So, we first iterate through the tooth coordinate information of all tooth frames and find the ordinate with the largest value, which is defined as the maximum ordinate. We then find the ordinate with the smallest value and define it as the minimum ordinate.
[0044] At this point, the minimum ordinate corresponds to the lowest position of the tooth region in the image, while the maximum ordinate represents the highest position of the tooth region in the image.
[0045] S320, calculate the median of the minimum and maximum ordinates, and define it as the longitudinal median.
[0046] Calculate the median between two vertical coordinates, which also corresponds to a vertical coordinate. This vertical median can be characterized as the ideal height median of the tooth region in the tooth image.
[0047] S330, determine whether the ordinate of the upper left corner of each tooth frame is greater than the median of the vertical axis.
[0048] Obtain the tooth coordinate information of the upper left corner of each tooth frame, select the vertical coordinate from the tooth coordinate information, and compare the vertical coordinate corresponding to the upper left corner of each tooth with the vertical median.
[0049] S340, if not greater than, then the tooth frame corresponds to the lower side of the tooth; if greater than, then calculate the vertical difference between the vertical coordinate of the upper left corner of the tooth frame and the vertical median, and determine whether the vertical difference is greater than the preset value.
[0050] If S350 is greater than this value, the tooth frame corresponds to the upper side of the tooth; if it is less than this value, the tooth frame corresponds to the lower side of the tooth.
[0051] First, if the ordinate of the top left corner of the tooth frame is less than or equal to the median, it means that the top of the tooth frame is still below or coincides with the ideal center position of the tooth image. Therefore, it must be a tooth frame located on the lower side of the tooth, and the tooth corresponding to this tooth frame must be the lower tooth.
[0052] Because the vertical median is selected as the ideal middle position of the teeth in the dental image, and the size of the upper and lower teeth of a person is different, and the size of the teeth corresponding to different people is also different, if the ordinate of the upper left corner of the tooth frame is greater than the vertical median, there may be two situations. The first situation is that the ordinate of the upper left corner of the upper tooth frame is greater than the vertical median. The other situation is that the patient's lower teeth are larger, causing the upper left corner of the lower tooth frame to be higher than the vertical median corresponding to the ideal middle position.
[0053] Then, it's necessary to re-evaluate the vertical difference between the ordinate of the upper left corner of the tooth frame and the vertical median. If the vertical difference is greater than a preset value (i.e., a large vertical difference), the tooth frame is identified as upper. If the vertical difference is smaller, the tooth frame is identified as lower. This is because the distance between the upper left corner of a tooth located on the upper side and the vertical median, which is the theoretical center position of the tooth, is always larger. Even if a lower tooth is larger than the ideal center position, the deviation will not be very large.
[0054] In other embodiments, during the determination of the longitudinal distribution information of teeth, the determination based on the upper left corner of the tooth frame can be changed to the upper right corner. If it is changed to the upper right corner, the overall determination method is not different.
[0055] In other embodiments, during the determination of the longitudinal distribution information of teeth, the determination based on the upper left corner of the tooth frame can also be changed to the lower left or lower right corner. If it is changed to the lower left or lower right corner, the above method is changed accordingly to: determine whether the ordinate corresponding to the lower left / lower right corner of each tooth frame is less than the longitudinal median. If it is not less than, the tooth frame corresponds to the upper side of the tooth. If it is less than, calculate the longitudinal difference between the ordinate corresponding to the lower left / lower right corner of the tooth frame and the longitudinal median. If the longitudinal difference is greater than a preset value, the tooth frame corresponds to the lower side of the tooth. If it is less than, the tooth frame corresponds to the upper side of the tooth.
[0056] The specific judgment logic and principle are the same as the judgment process based on the upper left corner point of the tooth frame.
[0057] In other embodiments, before obtaining the feature abscissas of tooth frames on the same side based on several tooth frames and obtaining the selected feature line based on the feature abscissas, the following steps are also included: Obtain user's missing tooth registration information and determine whether the user has missing teeth based on the user's missing tooth registration information.
[0058] If the user has missing teeth, the first selected method is used to obtain the feature x-coordinate, and the selected feature line is obtained based on the feature x-coordinate.
[0059] If the user does not have missing teeth, the second selected method is used to obtain the feature x-coordinate and the selected feature line is obtained based on the feature x-coordinate.
[0060] When uploading images of their teeth, users can also register and upload information about missing teeth at the same time. This allows the system to obtain information about the user's missing teeth and determine whether or not a tooth is missing based on the specific details in the registration information.
[0061] Users can register and upload information about missing teeth in various ways, including but not limited to: selecting the location of the missing tooth on the upper or lower side, left or right side, and starting with the left or right incisor as the initial tooth; generating a tooth image option where users can click on the missing tooth in the corresponding location; or directly entering the professional name of the missing tooth, etc.
[0062] Different feature x-axis selection methods are used depending on whether the user has missing teeth or not.
[0063] like Figure 3 As shown, in some other embodiments, if the user does not have missing teeth, the first selected method is used to obtain the feature abscissa and the selected feature line is obtained based on the feature abscissa, including the following steps: S410, if the user does not have missing teeth, then traverse all tooth frames located above the teeth to obtain the maximum and minimum horizontal coordinates, and use the maximum and minimum horizontal coordinates as feature horizontal coordinates.
[0064] After determining the vertical positional relationship of each tooth frame, find the maximum and minimum x-coordinates of all tooth frames located on the upper side of the teeth. According to the coordinate system in the tooth image, the tooth frame with the maximum x-coordinate corresponds to the rightmost tooth in the upper teeth, while the tooth frame with the minimum x-coordinate corresponds to the leftmost tooth in the upper teeth.
[0065] S420 generates a selected feature line with two feature x-coordinates as endpoints and parallel to the x-axis.
[0066] Generate a selected feature line. This selected feature line must satisfy the following conditions: the x-coordinates of its two endpoints correspond to the maximum and minimum x-coordinates of all tooth frames on the upper side of the tooth, respectively, and the selected feature line must be parallel to the x-axis. Therefore, the y-coordinates of all points on this selected feature line are the same.
[0067] Preferably, the ordinate of the selected feature point can be close to or equal to the longitudinal median. Since the longitudinal height of the upper incisor is generally more than half the height of the entire tooth, if the ordinate of the selected feature line is close to or equal to the longitudinal median, it can be ensured that the line will pass through the upper incisor and be located within the tooth frame corresponding to the upper incisor, which facilitates the subsequent determination of the left and right distribution of teeth.
[0068] In other embodiments, the tooth positioning reference position and corresponding reference coordinate information are obtained based on selected feature points, including the following steps: S510, obtain the midpoint of the selected feature line, use the midpoint as the tooth positioning reference, and obtain the coordinate information corresponding to the midpoint as the reference coordinate information.
[0069] Calculate the midpoint of the selected feature line and use it as a reference point for tooth positioning. Since the two endpoints of the selected feature line correspond to the outermost x-coordinate of the upper teeth, the selected feature line can be represented as a line segment representing the length of the upper teeth's arrangement. Therefore, the midpoint of this line segment can be represented as the ideal midpoint of the upper teeth's left-right arrangement in the tooth image. In the ideal tooth configuration, this midpoint will be located between the two incisors, allowing us to determine the left-right position of each tooth frame using this midpoint.
[0070] Since both the upper and lower teeth of a person are generally symmetrical or nearly symmetrical, when the midpoint generated based on the upper tooth frame is obtained, this midpoint can not only be identified as the ideal middle position of the left and right arrangement direction of the upper teeth, but also as the ideal middle position of the left and right arrangement direction of the lower teeth.
[0071] Meanwhile, in other embodiments, the selection of feature abscissas can also be achieved by traversing all tooth frames located on the underside of the tooth to obtain the maximum and minimum abscissas, and the selected feature line can be generated based on the maximum and minimum abscissas obtained through the tooth frames on the underside of the tooth.
[0072] However, since the number of upper teeth remains constant, but the lower teeth may be misaligned, some people may have one extra lower tooth on each side, or one extra lower tooth on one side. Therefore, obtaining the maximum and minimum abscissas through all the tooth frames located on the lower side of the teeth may result in the midpoint of the selected feature line not being in the ideal middle position. Therefore, it is preferable to obtain the maximum and minimum abscissas through the tooth frames located on the upper side.
[0073] The above method is applicable when the user does not have missing teeth. However, in other cases, if the patient has missing teeth, the missing teeth may correspond to the tooth frame where the maximum or minimum horizontal coordinate is located on the upper or lower side. If the outermost tooth on one side of the patient is missing, the midpoint selected by the above method cannot be used as the ideal middle position for the left and right arrangement of the teeth. Instead, it will be biased towards the left or right side of the teeth as a whole, which will have a certain impact on the subsequent judgment of the tooth position, especially the judgment of the position of the incisors.
[0074] like Figure 4 As shown, in some other embodiments, if the user has missing teeth, the second selection method is used to obtain the feature abscissa and the selected feature line is obtained based on the feature abscissa, including the following steps: S430, if the user has missing teeth, obtain the median x-coordinates of several tooth frames located on the upper side of the teeth, and define them as feature x-coordinates.
[0075] The median x-coordinate of each tooth frame can be calculated as (x_left + x_right) / 2, where x_left represents the x-coordinate of the upper left or lower left corner of the tooth frame, and x_right represents the x-coordinate of the upper right or lower right corner of the tooth frame.
[0076] Each median x-coordinate is defined as a characteristic x-coordinate.
[0077] S440, along the horizontal axis, generate selected feature lines that are parallel to the horizontal axis and have the feature coordinates corresponding to the target tooth frame and the feature coordinates corresponding to all other tooth frames located on the fixed horizontal axis side as endpoint coordinates.
[0078] The horizontal axis can be either the direction in which the horizontal coordinate increases or the direction in which the horizontal coordinate decreases.
[0079] For example, if the top left corner of the tooth image is taken as the origin of the coordinate system, then selecting the tooth frame along the direction of increasing horizontal coordinate means selecting each tooth frame as the target tooth frame from left to right in the tooth image. Conversely, selecting the tooth frame along the direction of decreasing horizontal coordinate means selecting each tooth frame as the target tooth frame from right to left in the tooth image.
[0080] The fixed horizontal axis direction is characterized by the horizontal axis direction that is identified and fixed based on the target tooth frame. For example, if the target tooth frame is selected along the direction of increasing horizontal coordinate, the fixed horizontal axis direction is characterized by the direction of decreasing horizontal coordinate, that is, from left to right.
[0081] When generating selected feature lines, for example, the target frame is first selected from the leftmost frame (the frame with the smallest x-coordinate). The x-coordinates of this frame and the x-coordinates of all the tooth frames to its right are then used as the x-coordinates of the two endpoints of the selected feature lines. For instance, if a person has 11 teeth on their upper side due to a missing tooth, then the leftmost tooth frame and the 10 tooth frames to its right will generate 10 selected feature lines. The second tooth frame on the left and the 9 tooth frames to its right will generate 9 selected feature lines, and so on, until a tooth frame has no corresponding tooth frames to its right, at which point the generation of selected feature lines ends.
[0082] In other embodiments, a left-right distribution queue of teeth is generated based on the tooth coordinate information and reference coordinate information corresponding to several tooth frames, including the following steps: S520: Obtain the midpoint of each selected feature line, and obtain the minimum discrete point group based on the midpoint.
[0083] The minimum discrete point set is characterized as the set of midpoints with the smallest degree of dispersion of the horizontal coordinate within a preset range.
[0084] In this embodiment of the application, the degree of dispersion can be understood as the difference between the horizontal coordinates corresponding to multiple midpoints. If, within the same horizontal coordinate range, the first set of midpoints includes 3 midpoints and the second set of midpoints contains 5 midpoints, then the degree of dispersion of the second set of midpoints is relatively small.
[0085] If a user has 11 upper teeth due to missing teeth, then the selected feature lines generated for the tooth outlines corresponding to these 11 teeth will total (10+9+8+7+6+5+4+3+2+1+0) = 55, resulting in 55 corresponding midpoints. Some of these selected feature lines may overlap.
[0086] Each midpoint represents the midpoint of the lateral distance between two tooth frames. The midpoint positions of each tooth frame and other tooth frames are relatively divergent. However, because human teeth are bilaterally symmetrical, each tooth frame must correspond to the same symmetrical position as a tooth frame in the same or similar position on the other side. For example, the symmetrical positions of the two incisors, the two canines, and the two lateral incisors should be the same or similar. This symmetrical position is located in the ideal middle position of the left and right arrangement of the teeth.
[0087] Therefore, after performing a dispersion analysis on these 55 midpoints, we will inevitably find that there are at most 6 midpoints with very small dispersion within a region. These 6 midpoints correspond to the symmetrical positions of the six pairs of symmetrical teeth on the left and right sides, respectively. These 6 midpoints then correspond to a set, and the elements in this set are the x-coordinates corresponding to these 6 midpoints.
[0088] S530: Obtain the x-coordinates of all midpoints within the smallest discrete point group and calculate the mean to generate the tooth positioning reference position and the corresponding reference coordinate information.
[0089] This set of midpoints corresponds to the ideal middle region in the left-right arrangement of the entire tooth. In order to further reduce the selection error of the ideal middle position, the mean of the abscissa of all midpoints in this minimum discrete point set is calculated, and this mean is used as the tooth positioning reference to obtain specific reference coordinate information.
[0090] Using the above method, even if a patient has missing teeth, no matter which one or several teeth the user is missing, the ideal midpoint can be found. This is because each tooth will be compared with several other teeth to find its midpoint. Therefore, there will always be several teeth that can be compared with symmetrical teeth, and the corresponding midpoint can be found through the symmetrical teeth.
[0091] It should be noted that, regardless of whether teeth are missing or not, the aforementioned tooth positioning reference can be a virtual horizontal coordinate, or a point or line generated based on that horizontal coordinate. If the tooth positioning reference is a point, the corresponding vertical coordinate is preferably close to or equal to the vertical median. If the tooth positioning reference is a line, all points on it have the same horizontal coordinate and pass through the upper and lower sides of the tooth, respectively.
[0092] In other embodiments, a left-right distribution queue of teeth is generated based on the tooth coordinate information and reference coordinate information corresponding to several tooth frames, including the following steps: S610, determine whether the x-coordinate corresponding to the reference coordinate information is located between the x-coordinates of the upper left and upper right corners of the tooth frame.
[0093] S620, if located, then determine the lateral distance between the tooth positioning reference point and the upper left and upper right corners of the tooth frame.
[0094] S630, if the horizontal distance between the upper left corner and the tooth positioning reference is greater than the horizontal distance between the upper right corner and the tooth positioning reference, the coordinate frame is located on the left side of the tooth; if the horizontal distance between the upper left corner and the tooth positioning reference is less than the horizontal distance between the upper right corner and the tooth positioning reference, the coordinate frame is located on the right side of the tooth.
[0095] Because the teeth on the left and right sides of a person are not perfectly symmetrical, and each tooth has a certain size difference, the tooth positioning reference position that is finally selected to represent the ideal middle position cannot be guaranteed to be located exactly between the two incisors. Therefore, the horizontal coordinate corresponding to the tooth positioning reference position will most likely fall within the horizontal coordinate range corresponding to the tooth frame of the left incisor or the horizontal coordinate range corresponding to the tooth frame of the right incisor.
[0096] If the x-coordinate corresponding to the reference coordinate information is between the x-coordinates of the upper left and upper right corners of the tooth frame, then the tooth frame represents the left or right incisor.
[0097] Because teeth are nearly centrally symmetrical, when there is a certain offset error in the selection of the tooth positioning reference position, the selected tooth positioning reference position is also relatively close to the middle position of the tooth. Therefore, if the tooth positioning reference position is located within the tooth frame of the left incisor, the horizontal coordinate of the tooth positioning reference position must be close to the horizontal coordinate corresponding to the right side of the tooth frame. Conversely, if the tooth positioning reference position is located within the tooth frame of the right incisor, the horizontal coordinate of the tooth positioning reference position must be close to the horizontal coordinate corresponding to the left side of the tooth frame.
[0098] Therefore, by determining the lateral distance between the tooth positioning reference point and the upper left and upper right corners of the tooth frame, it can be determined whether the tooth frame is a left-side tooth (left incisor) or a right-side tooth (right incisor).
[0099] S650: Obtain the upper left corner of each tooth frame located on the left side of the tooth, and generate a left queue sorting based on the difference in horizontal coordinates between each upper left corner and the tooth positioning reference position; obtain the upper right corner of each tooth frame located on the right side of the tooth, and generate a right queue sorting based on the difference in horizontal coordinates between each upper right corner and the tooth positioning reference position.
[0100] Once it is confirmed that each tooth frame is located to the left or right of the tooth, a queue order is generated based on the difference between the x-coordinate of the upper left or upper right corner of the tooth frame and the x-coordinate of the tooth positioning reference position. The queue order represents the arrangement sequence of each tooth.
[0101] For example, the greater the difference between the upper left corner of the tooth outline on the left side of the tooth and the horizontal coordinate of the tooth positioning reference, the closer the tooth outline is to the left side of the image; the smaller the difference, the closer it is to the middle position of the tooth in the image.
[0102] In other embodiments, if the abscissa corresponding to the reference coordinate information is not located between the abscissas of the upper left and upper right corners of the tooth frame, then the tooth frame whose abscissa is less than the abscissa corresponding to the reference coordinate information is determined to be located on the left side of the tooth, and the tooth frame whose abscissa is greater than the abscissa corresponding to the reference coordinate information is determined to be located on the right side of the tooth.
[0103] First, if there is no tooth positioning reference point between the upper left and upper right corners of a tooth frame, and since the tooth positioning reference point is the ideal middle position in the left-right arrangement direction of the teeth, then it means that the tooth frame must be located on the left or right side of the teeth.
[0104] Then, iterate through all the horizontal coordinates on the tooth frame. If all the horizontal coordinates are less than the horizontal coordinates corresponding to the reference coordinates, it means that the tooth frame must be located on the left side of the tooth as a whole. If all the horizontal coordinates are greater than the horizontal coordinates corresponding to the reference coordinates, it means that the tooth frame must be located on the right side of the tooth as a whole.
[0105] like Figure 5 As shown, in some other embodiments, several tooth frames are numbered based on the left-right distribution queue and the longitudinal distribution information of the teeth, including the following steps: First, classify the tooth frames corresponding to the upper teeth and the lower teeth according to the longitudinal distribution information of the teeth. For the upper teeth, the left teeth are numbered according to the obtained left-right distribution queue of the teeth, that is, the difference between the horizontal coordinate of the upper left corner of each tooth frame and the horizontal coordinate of the tooth positioning reference position. The teeth are numbered in ascending order of difference, with the smallest difference, which is the left incisor, numbered 11, and so on, numbered 12, 13, 14, 15, and 16 in the direction of increasing difference. The right teeth are numbered in ascending order of difference, with the smallest difference, which is the right incisor, numbered 21, and so on, numbered 22, 23, 24, 25, and 26 in the direction of increasing difference.
[0106] For the lower teeth, follow the same steps as above, numbering the tooth frames on the right side of the teeth as 31, 32, 33, 34, 35, 36, and the tooth frames on the left side of the teeth as 41, 42, 43, 44, 45, 46.
[0107] It should also be noted that the above method and step descriptions do not restrict the order of the numbers; the numbers can be defined and generated simultaneously.
[0108] Meanwhile, when a patient has missing teeth, the above numbering method and numbering order are not affected. If there is a certain distance and gap between two tooth frames due to missing teeth, the numbering will not jump from 22 to 24 due to missing teeth.
[0109] like Figure 6 As shown, in some other embodiments, numbering several tooth frames based on the left-right distribution queue and the longitudinal distribution information of teeth further includes the following steps: S800: If a user has missing teeth, the missing tooth number is obtained based on the user's missing tooth registration information.
[0110] S810, find the tooth number in the tooth image that is the same as the missing tooth number, and increment the number of the tooth number and the corresponding numbers of all tooth frames with a higher number by one.
[0111] When a user has missing teeth, the corresponding number of the missing tooth can be determined based on the information described, selected, or entered by the user in the missing tooth registration information.
[0112] After numbering each tooth in the dental image, the missing tooth number is entered into the corresponding tooth numbering system. Then, a number matching the missing tooth number is searched. For example, when the missing tooth number is 23, the number 23 is found in the corresponding tooth numbering system of each tooth in the dental image, along with all numbers greater than 23, namely 24 and 25 (since the upper right side is missing one tooth, 23, the total number of teeth on the upper right side is 5, so the number 26 does not exist after the initial tooth numbering). The numbers 23, 24, and 25 on the tooth frame are incremented by one, becoming 24, 25, and 26.
[0113] Using the above method, after initially numbering the tooth images, the numbers corresponding to the tooth frames on which side of the missing tooth is present can be updated according to the tooth loss situation, making the numbering of each tooth more accurate.
[0114] The implementation principle of this application embodiment is as follows: After users upload images of teeth using mobile devices such as smartphones, the system obtains the tooth frame and corresponding coordinates based on image processing. The system then uses the coordinates to directly determine the vertical position of the teeth and finds tooth positioning references that can be used to distinguish and compare the left and right positions of the teeth and to sort them to confirm the left and right distribution queue of the teeth. Finally, the system determines the tooth frame and the number of the teeth within the tooth frame by the vertical distribution information of the teeth corresponding to the entire tooth frame and the left and right distribution queue of the teeth. The numbering methods are all based on the processing and coordinate calculation of the user's two-dimensional image. They do not require professional dental imaging equipment or three-dimensional modeling. Users can directly generate the corresponding number information for the tooth image by uploading a tooth image, which is convenient for use in online consultations and AI consultations.
[0115] This application also discloses a real-time tooth numbering system, which is based on images captured by a mobile device, and is used to implement the above-mentioned real-time tooth numbering method.
[0116] The embodiments described in this specific implementation are preferred embodiments of this application and are not intended to limit the scope of protection of this application. Identical components are represented by the same reference numerals. Therefore, all equivalent changes made to the structure, shape, and principle of this application should be covered within the scope of protection of this application.
Claims
1. A real-time tooth numbering method, implemented based on images captured by a mobile device, characterized in that, Includes the following steps: Obtain images of teeth; The tooth image is processed to generate a tooth bounding box for each tooth and tooth coordinate information corresponding to several tooth bounding boxes; Based on the ordinate of each tooth coordinate, longitudinal distribution information of the teeth is generated, and the longitudinal distribution information of the teeth is used to characterize whether the tooth frame is located on the upper side or the lower side of the tooth. Select the feature abscissa of the tooth frame on the same side and obtain the selected feature line based on the feature abscissa, wherein the same side is characterized as both being on the upper side or both being on the lower side; Based on the selected feature line, the tooth positioning reference position and its corresponding reference coordinate information are obtained. The tooth positioning reference position is characterized as a reference object used to determine whether the tooth is located on the left or right side. A left-right distribution queue of teeth is generated based on the tooth coordinate information corresponding to several tooth frames and the reference coordinate information; The tooth frames are numbered based on the left-right distribution queue of the teeth and the longitudinal distribution information of the teeth.
2. The real-time tooth numbering method according to claim 1, characterized in that, The step of generating longitudinal distribution information of teeth based on the ordinate in the tooth coordinate information corresponding to each tooth frame specifically includes: Traverse the tooth coordinate information of the tooth frame to obtain the minimum and maximum ordinates; Calculate the median of the minimum and maximum ordinates, and define it as the longitudinal median; Determine whether the ordinate of the upper left corner of each tooth frame is greater than the median of the vertical axis; If it is not greater than, then the tooth frame is determined to correspond to the lower side of the tooth; if it is greater than, then the vertical difference between the vertical coordinate corresponding to the upper left corner of the tooth frame and the vertical median is calculated, and it is determined whether the vertical difference is greater than a preset value. If the value is greater than the value, the tooth frame is determined to correspond to the upper side of the tooth; if the value is less than the value, the tooth frame corresponds to the lower side of the tooth.
3. The real-time tooth numbering method according to claim 2, characterized in that, Before obtaining the feature abscissas of the tooth frames on the same side based on the several tooth frames and obtaining the selected feature line based on the feature abscissas, the method further includes the following steps: Obtain user's missing tooth registration information, and determine whether the user has missing teeth based on the user's missing tooth registration information; If the user does not have missing teeth, the first selected method is used to obtain the feature x-coordinate and the selected feature line is obtained based on the feature x-coordinate; If the user has missing teeth, the second selected method is used to obtain the feature x-coordinate and the selected feature line is obtained based on the feature x-coordinate.
4. The real-time tooth numbering method according to claim 3, characterized in that, If the user does not have missing teeth, the first selected method is used to obtain the feature abscissa, and the selected feature line is obtained based on the feature abscissa, including the following steps: If the user does not have any missing teeth, then all tooth frames located above the teeth are traversed to obtain the maximum and minimum horizontal coordinates, and the maximum and minimum horizontal coordinates are used as feature horizontal coordinates; Generate a selected feature line with the two described feature x-coordinates as endpoint x-coordinates and parallel to the x-axis.
5. The real-time tooth numbering method according to claim 4, characterized in that, Based on the selected feature lines, the tooth positioning reference position and its corresponding reference coordinate information are obtained. Includes the following steps: Obtain the midpoint of the selected feature line and use the midpoint as the tooth positioning reference. Obtain the coordinate information corresponding to the midpoint as the reference coordinate information.
6. The real-time tooth numbering method according to claim 3, characterized in that, If the user has missing teeth, the second selection method is used to obtain the feature abscissa and the selected feature line is obtained based on the feature abscissa, including the following steps: If the user has missing teeth, the median abscissa of several tooth frames located on the upper side of the teeth is obtained and defined as the feature abscissa. The median abscissa represents the median of the abscissas corresponding to the two sides of each tooth frame. Along the horizontal axis, each of the tooth frames is selected as the target tooth frame, and a selected feature line is generated with the feature horizontal coordinates corresponding to the target tooth frame and the feature horizontal coordinates corresponding to all other tooth frames located on the fixed horizontal axis side as the endpoint horizontal coordinates and parallel to the horizontal axis.
7. The real-time tooth numbering method according to claim 6, characterized in that, Based on the selected feature line, the tooth positioning reference position and its corresponding reference coordinate information are obtained, including the following steps: Obtain the midpoint of each of the selected feature lines, and obtain the minimum discrete point group based on several midpoints. The minimum discrete point group is characterized as the set of midpoints with the smallest degree of dispersion of the horizontal coordinate within a preset range. The abscissas of all the midpoints in the minimum discrete point group are obtained and the mean is calculated to generate the tooth positioning reference position and the corresponding reference coordinate information.
8. The real-time tooth numbering method according to claim 5 or 7, characterized in that, Generate a left-right distribution queue of teeth based on the tooth coordinate information corresponding to several tooth frames and the reference coordinate information, including the following steps: Determine whether the abscissa corresponding to the reference coordinate information is located between the abscissas of the upper left and upper right corners of the tooth frame; If it is located, then determine the lateral distance between the tooth positioning reference point and the upper left and upper right corner points of the tooth frame; If the lateral distance between the upper left corner and the tooth positioning reference is greater than the lateral distance between the upper right corner and the tooth positioning reference, then the coordinate frame is located on the left side of the tooth; if the lateral distance between the upper left corner and the tooth positioning reference is less than the lateral distance between the upper right corner and the tooth positioning reference, then the coordinate frame is located on the right side of the tooth. Obtain the upper left corner of each tooth frame located on the left side of the tooth, and generate a left queue sorting based on the difference in horizontal coordinate between each upper left corner and the tooth positioning reference position; obtain the upper right corner of each tooth frame located on the right side of the tooth, and generate a right queue sorting based on the difference in horizontal coordinate between each upper right corner and the tooth positioning reference position.
9. The real-time tooth numbering method according to claim 2, characterized in that, The method of numbering several tooth frames based on the left-right distribution queue and the longitudinal distribution information of the teeth further includes the following steps: If a user has missing teeth, the missing tooth number is obtained based on the user's missing tooth registration information; Find the tooth number in the tooth image that is the same as the missing tooth number, and add one to the number of that tooth number and the numbers of all tooth frames with a higher number.
10. A real-time tooth numbering system, implemented based on images captured by a mobile device, characterized in that, Used to implement the real-time tooth numbering method according to any one of claims 1-9.