Method, device and equipment for determining tooth color

Abstract

The embodiment of the invention provides a tooth color determining method, device and equipment. The reference pixel data of the sample teeth of each color number can be determined by using the images of the sample teeth of each color number in the shade guide collected by the three-dimensional scanning equipment. After a user selects a tooth to be colorimetric from a reconstructed three-dimensional model of the oral cavity of a patient, a target image frame of which the shooting distance and the shooting view angle meet a certain condition can be selected from a plurality of image frames which are acquired by a three-dimensional scanner and are used for reconstructing the three-dimensional model; and determining the color number of the to-be-colorimetric tooth based on the closeness of the pixel data of the to-be-colorimetric tooth in the target image frame and the pixel data of each color number sample tooth. According to the method, the target image frame with the shooting view angle and the shooting distance meeting certain conditions is selected from the image frames, so that the imaging quality of the to-be-colorimetric tooth in the target image frame can be ensured, and the accuracy and the reliability of a colorimetric result are improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of digital oral technology, and in particular to a method, device and apparatus for determining tooth color. Background Art

[0002] In the oral medical scenario, when repairing the patient's teeth, in order to ensure that the color of the restoration (such as porcelain teeth, veneers, etc.) is highly consistent with the color of the patient's natural teeth, it is usually necessary to compare the patient's natural teeth with the sample teeth in the color comparison plate to determine the color number of the patient's natural teeth. In the related art, either the color of the natural teeth and the color comparison plate is directly matched by the naked eye, and the color comparison results are easily affected by factors such as ambient lighting conditions and the experience of the comparer, and are not accurate enough and have poor repeatability. Or the color of natural teeth is matched by instruments such as spectrophotometers and colorimeters, but due to the complex shape of teeth, the accuracy of the color comparison results is poor. Therefore, it is necessary to provide a more accurate color comparison scheme. Summary of the invention

[0003] Embodiments of the present application provide a method, device and apparatus for determining tooth color.

[0004] According to a first aspect of an embodiment of the present application, a method for determining tooth color is provided, the method comprising: Determine the teeth to be compared in color from a three-dimensional model of the patient's oral cavity, wherein the three-dimensional model is reconstructed based on a plurality of image frames collected by a three-dimensional scanning device for the patient's oral cavity; A target image frame including the tooth to be compared in color is selected from the multiple image frames; wherein the shooting distance of the tooth to be compared in the target image frame is within a preset distance range and / or the shooting angle of the tooth to be compared in the target image frame deviates from a specified shooting angle by less than a preset degree, and the specified shooting angle is the shooting angle when the camera of the three-dimensional scanning device is facing the tooth to be compared in color; Acquire reference pixel data of sample teeth of each color number in the color chart, wherein the reference pixel data of the sample teeth of each color number is determined based on an image frame of the sample teeth of the color number acquired by the three-dimensional scanning device; The color of the tooth to be compared with the color is determined based on the proximity between the pixel data of the tooth to be compared with the color in the target image frame and the reference pixel data of the sample teeth of each color number.

[0005] According to a second aspect of an embodiment of the present application, a device for determining tooth color is provided, the device comprising: A module for determining teeth to be compared in color, used to determine teeth to be compared in color from a three-dimensional model of the patient's oral cavity, wherein the three-dimensional model is reconstructed based on a plurality of image frames collected by a three-dimensional scanning device for the patient's oral cavity; A selection module, for selecting a target image frame including the tooth to be compared in color from the multiple image frames; wherein the shooting distance of the tooth to be compared in the target image frame is within a preset distance range and / or the shooting angle of the tooth to be compared in the target image frame deviates from a specified shooting angle by less than a preset degree, and the specified shooting angle is the shooting angle of the three-dimensional scanning device facing the tooth to be compared in color; An acquisition module, used for acquiring reference pixel data of sample teeth of each color number in the color chart, wherein the reference pixel data of the sample teeth of each color number is determined based on an image frame of the sample teeth of the color number acquired by the three-dimensional scanning device; The colorimetric module is used to determine the color of the tooth to be compared based on the proximity between the pixel data of the tooth to be compared in the target image frame and the reference pixel data of the sample teeth of each color number.

[0006] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a processor, a memory, and computer instructions stored in the memory and executable by the processor, wherein when the processor executes the computer instructions, the method mentioned in the first aspect above can be implemented.

[0007] According to a fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided, on which computer instructions are stored. When the computer instructions are executed, the method mentioned in the first aspect is implemented.

[0008] In the embodiment of the present application, a three-dimensional scanning device can be used to scan the sample teeth of each color number in the color comparison plate to acquire images of each sample tooth, and the pixel data of the sample teeth of each color number can be determined based on these images as reference pixel data. After the user selects the tooth to be compared in color from the reconstructed three-dimensional model of the patient's oral cavity, a target image frame whose shooting distance and shooting angle meet certain conditions can be selected from multiple image frames acquired by the three-dimensional scanner for reconstructing the three-dimensional model, and the color number of the tooth to be compared is determined based on the degree of proximity between the pixel data of the tooth to be compared in the target image frame and the pixel data of the sample teeth of each color number. In the embodiment of the present application, the position information of each image frame captured by the camera obtained during the three-dimensional reconstruction process, as well as the depth information of each image frame, can be used to select a target image frame whose shooting angle and shooting distance meet certain conditions from the image frame, thereby ensuring the imaging quality of the tooth to be compared in the target image frame and improving the accuracy and reliability of the color comparison result.

[0009] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory and are not restrictive of the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the embodiments of the present application and, together with the specification, are used to illustrate the technical solutions of the embodiments of the present application.

[0011] Figure 1 It is a schematic diagram of a colorimetric plate.

[0012] Figure 2 It is a schematic diagram of a method for determining tooth color according to an embodiment of the present application.

[0013] Figure 3 The figure is a flow chart of a method for determining tooth color according to an embodiment of the present application.

[0014] Figure 4 It is a schematic diagram of an application scenario of an embodiment of the present application.

[0015] Figure 5 It is a schematic diagram of selecting a target image frame according to an embodiment of the present application.

[0016] Figure 6 This is a schematic diagram of dividing teeth into regions according to an embodiment of the present application.

[0017] Figure 7 It is a schematic diagram of the logical structure of a device for determining tooth color according to an embodiment of the present application.

[0018] Figure 8 It is a schematic diagram of the logical structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0019] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the appended claims.

[0020] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, but are not intended to limit the embodiments of the present application. The singular forms of "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include a plurality of forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a variety of.

[0021] It should be understood that, although the terms first, second, third, etc. may be used to describe various information in the embodiments of the present application, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the embodiments of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determination".

[0022] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application and to make the above-mentioned purposes, features and advantages of the embodiments of the present application more obvious and understandable, the technical solutions in the embodiments of the present application are further described in detail below in conjunction with the accompanying drawings.

[0023] In oral medical scenarios, when repairing a patient's teeth, in order to ensure that the color of the restoration (such as porcelain teeth, veneers, etc.) is highly consistent with the patient's natural teeth and improve the aesthetics of the restoration, it is usually necessary to compare the patient's natural teeth with the sample teeth in the color comparison plate to determine the color number of the patient's natural teeth.

[0024] For example, Figure 1 The figure shows a schematic diagram of a colorimetric plate, which includes a plurality of sample teeth, each of which corresponds to a color number. When determining the color number of a patient's natural teeth, the patient's teeth and the sample teeth in the colorimetric plate can be color matched to determine the color number of the patient's natural teeth, and a dental restoration can be subsequently made based on the color number.

[0025] The traditional colorimetric method is visual colorimetry, which is to match the color of the color plate with the color of the natural tooth directly by the naked eye. This colorimetric method is easily affected by factors such as doctor experience, ambient light conditions, and human visual fatigue, resulting in unstable colorimetric results and poor repeatability. With the development of digital technology, instrumental colorimetry has gradually been applied to clinical practice. Instruments such as colorimeters and spectrophotometers can reduce the interference of human factors to a certain extent, but these instruments have limitations when measuring teeth of complex shapes. The measurement results are affected by the unevenness of the tooth surface and volume reflection, resulting in insufficient colorimetric accuracy and reliability. Therefore, it is necessary to provide a more accurate tooth colorimetric solution.

[0026] Based on this, the embodiment of the present application provides a method for determining tooth color. Considering that oral digital impression technology has been widely used in clinical practice, a three-dimensional model of the patient's oral cavity can be quickly and efficiently obtained by scanning the patient's oral cavity with a three-dimensional scanning device. In order to improve the accuracy and efficiency of scanning, the three-dimensional scanning device usually adopts advanced optical imaging technology, such as blue light scanning or confocal microscopy technology, which can accurately extract the color information of the patient's teeth. Therefore, the applicant thinks that the color of the patient's teeth can be determined with the help of a three-dimensional scanning device.

[0027] For example, Figure 2 As shown, a three-dimensional scanning device can be used to scan the sample teeth of each color number in the color comparison plate to acquire images of each sample tooth, and the pixel data of the sample teeth of each color number can be determined based on these images as reference pixel data. In addition, when a three-dimensional reconstruction of the patient's oral cavity is performed, it is usually necessary to acquire multiple frames of images of the patient's oral cavity from different perspectives. Therefore, the pixel data of the teeth to be compared can be determined with the help of these images, and then the color number of the teeth to be compared can be determined based on the degree of proximity between the pixel data of the teeth to be compared and the reference pixel data of the sample teeth of each color number. Further, considering that when a three-dimensional scanning device is used to scan the patient's oral cavity, the oral cavity includes the tongue, saliva, etc., and the environment in the oral cavity is relatively complex, the acquired image is easily affected by the environment in the oral cavity, resulting in unstable imaging quality of the acquired image. For example, when shooting the same tooth, the shooting angle, shooting distance, etc. will affect the exposure, reflection and color cast of the image. Therefore, when determining the pixel data of the teeth to be compared based on the images captured by the three-dimensional scanning device, a target image frame whose shooting angle and / or shooting distance meet certain conditions can be selected to ensure the imaging quality of the selected target image frame and improve the accuracy of the color comparison results.

[0028] The method for determining tooth color provided in the embodiment of the present application can be executed by various electronic devices. For example, the electronic device can be a three-dimensional scanning device, or other devices that are communicatively connected to the three-dimensional scanning device, such as a mobile phone, a tablet, a notebook, a cloud server, etc., and the embodiment of the present application does not limit this.

[0029] like Figure 3As shown, it is a schematic diagram of an application scenario of the present application. In some embodiments, the method can be executed by other devices that are connected to the three-dimensional scanning device in communication. The other device can be installed with scanning software, and the user can set scanning parameters through the scanning software. At the same time, the three-dimensional scanning device can send the collected image of the patient's mouth to the scanning software, so that the scanning software can reconstruct the three-dimensional model of the patient's mouth and display it to the user. Among them, the scanning software can integrate a color comparison function, and the user can select the teeth to be compared from the reconstructed three-dimensional model. The scanning software can execute the method to determine the color number of the teeth to be compared and display it to the user.

[0030] Specifically, Figure 4 As shown, the method for determining tooth color provided in the embodiment of the present application may include the following steps: S402, determining the teeth to be compared in color from a three-dimensional model of the patient's oral cavity, wherein the three-dimensional model is reconstructed based on a plurality of image frames collected by a three-dimensional scanning device for the patient's oral cavity; In step S402, a 3D scanning device may be used to scan the patient's oral cavity to acquire multiple image frames of the patient's oral cavity, and then the patient's oral cavity may be 3D reconstructed based on the multiple image frames to obtain a 3D model of the patient's oral cavity. When tooth color comparison is required, the teeth to be compared may be determined from the 3D model of the patient's oral cavity.

[0031] There are many ways to determine the teeth to be compared in color from the three-dimensional model of the patient's oral cavity. The method can be determined manually by the user or automatically, and the specific method can be set based on actual needs.

[0032] For example, in some embodiments, the three-dimensional model can be displayed in the interactive interface, and in response to the user's operation of selecting the tooth to be compared, the tooth to be compared is determined from the three-dimensional model of the patient's oral cavity. Considering that the tooth to be compared is usually a tooth that needs to be repaired, or a tooth adjacent to the tooth to be repaired, for example, a tooth located to the left, right, above, or below the tooth to be repaired. Therefore, the three-dimensional model of the patient's oral cavity can be displayed to the user so that the user can determine the tooth to be compared based on the three-dimensional model.

[0033] There are many ways for users to select teeth to be compared in color. For example, users can directly click and select a tooth as the tooth to be compared in the three-dimensional model displayed on the interactive interface, or users can input the tooth position number of the tooth to be compared in color through the interactive component, so as to determine the tooth to be compared in color from the three-dimensional model based on the tooth position number, or users can input voice commands, so as to determine the tooth to be compared in color based on the user's voice commands.

[0034] In some embodiments, a pre-trained AI model may be used to identify the tooth to be repaired from the three-dimensional model, and then the tooth to be repaired or a tooth adjacent to the tooth to be repaired is used as the tooth to be compared in color.

[0035] In some embodiments, the user can also input an identity recognition instruction, log in to the user's personal account, and then retrieve historical information related to the user, such as the patient's medical history recorded by the user, the information marked by the user in the three-dimensional model, etc., determine the tooth to be repaired based on the historical information, and then use the tooth to be repaired or the adjacent tooth to be repaired as the tooth to be compared with the color.

[0036] Among them, the three-dimensional scanning device can be an oral scanner, an extraoral scanner, a facial scanner, an electronic colorimeter and other equipment. As long as the three-dimensional model of the patient's oral cavity can be reconstructed based on the data collected by the device, the embodiments of the present application are not limited.

[0037] S404, selecting a target image frame including the tooth to be compared in color from the multiple image frames; wherein the shooting distance of the tooth to be compared in the target image frame is within a preset distance range and / or the shooting angle of the tooth to be compared in the target image frame deviates from a specified shooting angle by less than a preset degree, and the specified shooting angle is the shooting angle when the camera of the three-dimensional scanning device is facing the tooth to be compared in color; In step S404, after the teeth to be compared are determined, the target image frame containing the teeth to be compared can be determined from the multiple image frames collected by the three-dimensional scanning device. At the same time, it is considered that factors such as the shooting angle and the shooting distance will affect the exposure area and the reflective area of ​​the imaged image, thereby causing a certain degree of color cast in the teeth to be compared. In order to ensure the accuracy of color comparison, the target image frame whose shooting angle and shooting distance meet certain conditions can be selected from the multiple image frames collected by the three-dimensional scanning device to ensure the imaging quality of the teeth to be compared in the target image frame. In the process of three-dimensionally reconstructing the patient's oral cavity using the image of the patient's oral cavity collected by the three-dimensional scanning device, the posture of each frame of the image taken by the camera can be determined, and the depth information of each pixel in each frame of the image can also be determined. Therefore, these information in the three-dimensional reconstruction process can be reused to select the target image frame whose shooting angle and shooting distance meet the conditions from the multiple image frames collected by the three-dimensional scanning device.

[0038] For example, Figure 5As shown, after the three-dimensional model of the patient's oral cavity is displayed on the interactive interface, the user can select the tooth to be compared in the three-dimensional model, and then use the tooth to be compared as a reference to select a target image frame whose shooting angle and shooting distance meet the conditions from the multiple image frames collected, wherein the target image frame can be one or more frames. By setting the shooting angle and shooting distance as constraints, the target image frame that meets the requirements can be screened out from the video stream collected by the three-dimensional scanning device, which can ensure the accuracy and consistency of the color comparison results.

[0039] Among them, the selected target image frame can meet one or more of the following conditions: the shooting distance of the teeth to be compared in the target image frame is within a preset distance range, and the deviation degree between the shooting angle of the teeth to be compared in the target image frame and the specified shooting angle is less than a preset degree, wherein the specified shooting angle is the shooting angle of the camera of the three-dimensional scanning device when it is facing the teeth to be compared.

[0040] The preset distance range can be determined based on the hardware parameters of the three-dimensional scanning device and the test data obtained from multiple tests. For example, the preset distance range can be determined based on the depth of field range of the camera in the three-dimensional scanning device to ensure that the image of the teeth to be compared is clear. Alternatively, a suitable distance range can be determined through the test data obtained from multiple tests, so that when the three-dimensional scanning device takes a picture of the teeth to be compared within the distance range, the image quality of the teeth to be compared is high, the color distortion is small, etc.

[0041] In addition, considering that when a camera is facing an object and taking a picture of the object, the image quality of the object in the captured image is better and color distortion is less likely to occur. When the shooting angle deviates from the angle when facing the object, the greater the deviation, the worse the imaging effect of the object. Therefore, when selecting a target image frame, the deviation degree between the shooting angle of the tooth to be compared in the selected target image frame and the specified shooting angle is less than a preset degree, wherein the preset degree can also be determined based on the hardware parameters of the three-dimensional scanning device and the test data obtained from multiple tests, so as to ensure that the imaging effect of the tooth to be compared in the target image frame where the deviation degree between the shooting angle and the specified shooting angle is within the preset degree is better.

[0042] S406, obtaining reference pixel data of the sample teeth of each color number in the color chart, wherein the reference pixel data of the sample teeth of each color number is determined based on the image frame of the sample teeth of the color number collected by the three-dimensional scanning device; In step S406, reference pixel data of the sample teeth of each color number in the predetermined color chart can be obtained. For example, for the sample teeth of each color number in the color chart, the sample teeth can be scanned using a three-dimensional scanning device to obtain one or more frames of images, and then the pixel data of the sample teeth can be determined based on the one or more frames of images as reference pixel data. The reference pixel data includes the pixel values ​​of each pixel channel of the sample teeth in the target color space. For example, taking the target color space as the LAB color space as an example, for the sample teeth of each color number, the pixel value of the sample teeth in the L channel, the pixel value of the A channel, and the pixel value of the B channel can be determined. The pixel value of each channel can be the mean or center value of the pixel values ​​of multiple pixel points in the image area corresponding to the sample teeth in the image frame.

[0043] S408: Determine the color of the tooth to be compared based on the proximity between the pixel data of the tooth to be compared in the target image frame and the reference pixel data of the sample teeth of various color numbers.

[0044] In step S408, if the colors of two objects are close, then the pixel values ​​of the image areas corresponding to the two objects are also close. Therefore, after obtaining the target image frame and the reference pixel data of the sample teeth of various color numbers, the color of the tooth to be compared can be determined based on the pixel data of the tooth to be compared in the target image frame and the degree of proximity to the reference pixel data of the sample teeth of various color numbers. After determining the color of the tooth to be compared, the prompt information related to the color can be displayed in the interactive interface, for example, the color number corresponding to the color can be displayed.

[0045] Considering that the reference pixel data is used as a benchmark for comparison, in order to make the reference pixel data more accurately and truly reflect the color of the model teeth under different acquisition environments, in some embodiments, when determining the reference pixel data of the model teeth of each color number, a three-dimensional scanning device can be used to scan the model teeth of the color number from different angles and / or different distances to obtain multiple image frames of the model teeth, so that the multiple image frames collected can cover various shooting angles and shooting distances. For each pixel channel, the mean or center value of the pixel data of the model teeth in the channel in the multiple image frames can be used as the reference pixel data of the model teeth in the pixel channel. For example, as shown in the figure, taking the reference pixel data as the pixel data in the LAB color space as an example, for the L channel, since the multiple image frames cover different shooting angles and shooting distances, the pixel values ​​of the L channel of the model teeth in the multiple image frames can reflect the brightness distribution of the model teeth under different shooting conditions, and then the center or mean of the pixel values ​​of the L channel can be obtained as the reference pixel value of the model teeth in the L channel. For other pixel channels, a similar method can be adopted to determine the reference pixel values ​​of the sample teeth in other pixel channels. In this way, scanning errors can be eliminated, and the determined reference data can more accurately reflect the real pixel values ​​of the sample teeth.

[0046] In some embodiments, the selected target image frames include multiple ones. When determining the color of the tooth to be compared based on the pixel data of the tooth to be compared in the target image frame and the proximity of the reference pixel data of the sample teeth of each color number, the pixel data of the tooth to be compared in the multiple target image frames can be first fused to obtain fused pixel data, and then the fused pixel data can be compared with the reference pixel data of the sample teeth of each color number. Considering that the imaging quality and imaging effect of the tooth to be compared in different target image frames are different, in order to obtain pixel data that can accurately reflect the color information of the tooth to be compared, when the pixel data of the tooth to be compared in the multiple target image frames are fused, the weight corresponding to each target image frame can be determined first, wherein the weight corresponding to each target image frame can be determined based on the shooting distance and / or shooting angle of the tooth to be compared in the target image frame. Then, the pixel data of the tooth to be compared in the multiple target image frames can be weighted averaged based on the weight corresponding to each target image frame to obtain fused pixel data. The fused pixel data can then be compared with the reference pixel data of the sample teeth of each color number to obtain the degree of proximity between the fused pixel data and the reference pixel data of the sample teeth of each color number, and the color number of the sample tooth with the greatest degree of proximity is used as the color number of the tooth to be compared.

[0047] In some embodiments, if the shooting distance of the teeth to be compared in the target image frame deviates more from the specified distance range, the weight corresponding to the target image frame is smaller. The specified distance range can be a distance range set based on the hardware parameters and experience values ​​of the three-dimensional scanning device. When the shooting distance is within the distance range, the teeth to be compared can be clearly imaged and the color of the teeth to be compared is less likely to be distorted.

[0048] In some embodiments, if the tooth to be compared in the target image frame deviates more from the specified shooting angle, the weight corresponding to the target image frame is smaller. The specified shooting angle is the shooting angle when the camera of the three-dimensional scanning device is facing the tooth to be compared. Usually, when the camera is facing an object and shooting the object, the image quality of the object in the image captured by the camera is better, and the color is less likely to be distorted or cast. Therefore, for each target image frame, the deviation between the shooting angle of the tooth to be compared in the target image frame and the angle when the camera is facing the tooth to be compared can be determined. If the deviation is larger, the weight corresponding to the target image frame is smaller.

[0049] In some embodiments, the selected target image frames include multiple ones. When determining the color of the tooth to be compared based on the proximity between the pixel data of the tooth to be compared in the target image frame and the reference pixel data of the sample teeth of each color number in the color comparison plate, for each target image frame, the pixel data of the tooth to be compared in the target image frame can be compared with the reference pixel data of the sample teeth of each color number to determine the proximity between the two, and then the color number of the tooth to be compared with the highest degree of proximity can be taken as the color number corresponding to the target image frame, so that the color number corresponding to the tooth to be compared in each target image frame can be obtained. Then, the number of target image frames corresponding to each color number can be counted, and the color number with the largest number can be taken as the color number of the tooth to be compared.

[0050] For example, suppose that there are 5 target image frames selected, which are marked as frame 1, frame 2, frame 3, frame 4 and frame 5. For each target image frame, the pixel data of the tooth to be compared in the frame is compared with the reference pixel data of the sample teeth of each color number in the color plate to determine the color number corresponding to the tooth to be compared in each target image frame. Assume that the color number corresponding to the tooth to be compared in each target image frame is as follows: Frame 1: color number A; Frame 2: color number C; Frame 3: color number B; Frame 4: color number A; Frame 5: color number A; Count the number of target image frames corresponding to each color number: Color number A: Frame 1, Frame 4 and Frame 5, 3 frames in total; Color number B: frame 3, total 1 frame; Color number C: frame 2, 1 frame in total; The statistical results show that the number of image frames corresponding to color number A is the largest, so the color of the tooth to be compared is finally determined to be color number A.

[0051] The above embodiment reduces the error that may be caused by a single frame image through the comparison and statistics of multiple frame images, thereby improving the accuracy and reliability of tooth color matching.

[0052] Considering that the colors of different areas of the same tooth may be different, in order to more accurately determine the color of the tooth, in some embodiments, when determining the color of the tooth to be compared based on the pixel data of the tooth to be compared in the target image frame and the proximity of the reference pixel data of the sample teeth of each color number in the color plate, for each target image frame, the tooth to be compared in the target image frame can be divided into multiple tooth areas, and for each tooth area, the pixel data of the tooth area can be compared with the reference data of the sample teeth of each color number to determine the proximity between the two, and then the color number of the sample tooth with the highest degree of proximity is taken as the color number of the tooth area. Then, the color number of the tooth to be compared can be determined based on the color numbers of each of the multiple tooth areas. For example, a tooth can be divided into 9 squares to obtain 9 areas. Through the above comparison process, it is determined that the color number of 5 tooth areas among the 9 tooth areas is color number A, the color number of 2 tooth areas is color number B, and the color number of 2 tooth areas is color number C. Therefore, the color number of the tooth can be determined to be color number A.

[0053] In the above embodiment, by dividing the teeth into regions and then performing comparison based on the regions, the tooth color can be determined more accurately, which is particularly suitable for scenes where the color distribution of teeth is uneven.

[0054] In some embodiments, the multiple tooth regions include a neck, a body, and an incisor. When determining the color number of the tooth to be compared based on the color numbers of the multiple tooth regions, the color number of the tooth to be compared can be determined based on the weights corresponding to the multiple tooth regions and the color numbers of the multiple tooth regions, wherein the weight of the body is higher than the weights of the neck and the incisor. Figure 6As shown, teeth can usually be divided into three parts: the incisal tip, the body, and the neck. The neck is located at the bottom of the tooth and in contact with the gums. Its enamel is thin, and the color of the dentin can be seen, and the color saturation of the tooth is large. The body is the middle part of the tooth, which is usually easier to see. Therefore, clinical color comparison usually uses the body as the main color of natural teeth. The incisal tip is located at the top of the tooth, with thick enamel and thin dentin. The color of the tooth is mainly due to the translucency and opalescence effect of the enamel. In order to prepare a restoration with a better aesthetic effect, weights can be set for each tooth area based on the characteristics of each tooth area, and the color number of the tooth to be compared can be determined based on the weight of each tooth area and the color number of each tooth area. Among them, considering that the body is located in the middle part of the tooth and is easier to see, the weight of the body can be higher than the weights of the neck and the incisal tip.

[0055] For example, assuming that there are 5 target image frames, each target image frame can be divided into the neck, body, and incisal end, and the color number of each tooth area can be determined, assuming the following: Frame 1: Neck, color A; body, color B; incisal end, color B Frame 2: Neck, color A; body, color B; incisal end, color C Frame 3: Neck, color B; body, color B; incisal end, color C Frame 4: Neck, color A; body, color A; incisal end, color C Frame 5: Neck, color A; body, color B; incisal end, color C Among them, color number A: 4 times in the neck, 1 time in the body, and 0 times in the incisal end; Color B: 1 time on the neck, 4 times on the body, and 1 time on the incisal end; Color code C: 0 times in the neck, 0 times in the body, 4 times in the incisal end; Assume that the weight of the neck is 0.2, the weight of the body is 0.5, and the weight of the incisal end is 0.3. The final scores of each color number are as follows: Score of color number A: 4×0.2+0.5×1+0=1.3 Score of color number B: 1×0.2+0.5×4+1×0.3=2.5 Score for color number C: 4×0.3=1.2 Shade B has the highest score, so it can be determined that the color of the tooth to be compared is shade B.

[0056] In some embodiments, the pixel data of the tooth to be compared and the reference pixel data are both pixel data in the target color space. When determining the proximity between the pixel data of the tooth to be compared in each target image frame and the pixel data of the sample tooth of each color number, for any pixel channel in the target color space, the proximity between the pixel data of the tooth to be compared in the target image frame in the pixel channel and the reference pixel data of the sample tooth of the color number in the corresponding pixel channel can be determined. Then, based on the weight of each pixel channel, the corresponding proximity of multiple pixel channels in the target color space is weighted averaged to obtain the final proximity. The weights of different pixel channels are determined based on the sensitivity of the pixel value of the pixel channel to the color number change. The more sensitive the pixel channel, the greater its weight. Among them, the target color space can be various color spaces, for example, it can be LAB color space, RGB color space, HSV color space, etc. Considering that the pixel values ​​of different pixel channels are not the same in sensitivity to the color number change, for example, taking the target color space as LAB color space, for the sample teeth with color numbers B1, B2, and B3, as the color number changes, the pixel values ​​of the L channel and A channel of the sample teeth in the LAB color space do not change much, but the pixel value of the B channel will continue to increase, that is, the color number change has a greater impact on the pixel value of the B channel. In order to better distinguish different color numbers and accurately determine the color of the teeth to be compared, when comparing pixel data, the weights of the pixel values ​​of different pixel channels can be set differently, among which the weights of the pixel channels that are sensitive to color number changes can be set larger.

[0057] In some embodiments, the pixel data of the teeth to be compared and the reference pixel data are both pixel data in the LAB color space. The LAB color space is a color model based on human visual perception, which can more accurately reflect the information of the three dimensions of color brightness (L), chromaticity (a, b). Compared with the traditional RGB color space, the Lab color space is less dependent on lighting conditions (i.e., less affected by lighting conditions), and can more accurately measure and describe tooth color. By comparing the pixel data of the teeth to be compared with the reference pixel data of sample teeth of various color numbers in the LAB color space, the accuracy and stability of the color comparison can be improved, and a more accurate color comparison result can be obtained.

[0058] In some embodiments, after determining the color of the tooth to be compared, the color number of the tooth to be compared can also be displayed in the interactive interface. For example, the color number can be displayed near the tooth to be compared in the interactive interface. For example, taking the method performed by scanning software as an example, the user can move the cursor to different teeth in the three-dimensional model. When the cursor stays on a certain tooth, the scanning software can determine the color number of the tooth by the above method, and then display the color number near the tooth. For example: when the cursor stays on a certain tooth, the scanning software determines the tooth position number and the color number of the tooth, displays the tooth position number and the color number simultaneously in a certain area of ​​the screen, generates a corresponding table, or retrieves and displays a picture of the color number in the color comparison plate.

[0059] In some embodiments, in order to allow the user to more intuitively see the effect of the color on the teeth, a three-dimensional model of the teeth rendered with the color may also be directly displayed on the interactive interface.

[0060] In some embodiments, when determining the color of the tooth to be compared based on the degree of proximity between the pixel data of the tooth to be compared in the target image frame and the reference pixel data of the sample teeth of each color number, the three color numbers whose reference pixel data are most similar to the pixel data of the tooth to be compared in the target image frame can be determined, and then the three color numbers can be displayed in the interactive interface.

[0061] The tooth color comparison scheme provided in the embodiment of the present application can realize full-process automated processing, which significantly improves the convenience of operation compared with traditional manual color comparison methods such as endoscopes. In addition, when determining the reference pixel data of the sample teeth and the pixel data of the teeth to be compared, the scheme can be based on multiple frames of images collected from different viewing angles or different shooting distances by a three-dimensional scanning device. By fusing the scanning results of multiple viewing angles, a more stable and reliable color comparison result can be provided. In addition, compared with the color comparison scheme based on the AI ​​model, this scheme does not require a large amount of training data, especially some data with relatively scarce tooth colors. In addition, the portability of the scheme is also relatively high, and it can be quickly migrated to various different color comparison plates.

[0062] It is not difficult to understand that the solutions described in the above embodiments can be freely combined to obtain new solutions when there is no conflict. Due to space reasons, they are not listed one by one in the embodiments of this application.

[0063] Accordingly, the embodiment of the present application also provides a device for determining tooth color, such as Figure 7 As shown, the device 70 includes: A tooth determination module 71 for color matching is used to determine the tooth to be color matched from a three-dimensional model of the patient's oral cavity, wherein the three-dimensional model is reconstructed based on a plurality of image frames collected by a three-dimensional scanning device for the patient's oral cavity; A selection module 72 is used to select a target image frame containing the tooth to be compared in color from the multiple image frames; wherein the shooting distance of the tooth to be compared in the target image frame is within a preset distance range and / or the shooting angle of the tooth to be compared in the target image frame deviates from a specified shooting angle by less than a preset degree, and the specified shooting angle is the shooting angle of the tooth to be compared when the three-dimensional scanning device is facing the tooth to be compared in color; An acquisition module 73 is used to acquire reference pixel data of the sample teeth of each color number in the color chart, wherein the reference pixel data of the sample teeth of each color number is determined based on the image frame of the sample teeth of the color number collected by the three-dimensional scanning device; The colorimetric module 74 is used to determine the color of the tooth to be compared based on the proximity between the pixel data of the tooth to be compared in the target image frame and the reference pixel data of the sample teeth of various color numbers.

[0064] Among them, the specific steps of the above-mentioned device executing the task processing method can refer to the description in the above-mentioned method embodiment, and will not be repeated here.

[0065] Furthermore, the present application also provides an electronic device, such as Figure 8 As shown, the device includes a processor 81, a memory 82, and computer instructions stored in the memory 82 and executable by the processor 81. When the processor 81 executes the computer instructions, the method described in any one of the above embodiments is implemented.

[0066] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any of the foregoing embodiments.

[0067] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined in this article, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0068] It can be known from the description of the above implementation mode that a person skilled in the art can clearly understand that the embodiment of the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the embodiment of the present application or some parts of the embodiment.

[0069] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver, a game console, a tablet computer, a wearable device or a combination of any of these devices.

[0070] Each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts refer to the partial description of the method embodiment. The device embodiment described above is merely schematic, wherein the modules described as separate components may or may not be physically separated, and the functions of each module can be implemented in the same one or more software and / or hardware when implementing the embodiment scheme of the present application. It is also possible to select some or all of the modules according to actual needs to achieve the purpose of the embodiment scheme. Those of ordinary skill in the art can understand and implement it without paying creative work.

[0071] The above is only a specific implementation of the embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the embodiment of the present application. These improvements and modifications should also be regarded as the protection scope of the embodiment of the present application.

Claims

1. A method for determining tooth color, characterized in that: The method comprises: Determine the teeth to be compared in color from a three-dimensional model of the patient's oral cavity, wherein the three-dimensional model is reconstructed based on a plurality of image frames collected by a three-dimensional scanning device for the patient's oral cavity; A target image frame including the tooth to be compared in color is selected from the multiple image frames; wherein the shooting distance of the tooth to be compared in the target image frame is within a preset distance range and / or the shooting angle of the tooth to be compared in the target image frame deviates from a specified shooting angle by less than a preset degree, and the specified shooting angle is the shooting angle when the camera of the three-dimensional scanning device is facing the tooth to be compared in color; Acquire reference pixel data of sample teeth of each color number in the color chart, wherein the reference pixel data of the sample teeth of each color number is determined based on an image frame of the sample teeth of the color number acquired by the three-dimensional scanning device; The color of the tooth to be compared with the color is determined based on the proximity of the pixel data of the tooth to be compared with the color in the target image frame to the reference pixel data of the sample teeth of each color number.

2. The method according to claim 1, characterized in that The reference pixel data of the sample teeth for each shade is determined based on the following method: Scanning the sample teeth of the color number from different angles and / or different distances using the three-dimensional scanning device to obtain multiple image frames of the sample teeth; For each pixel channel, the mean value or the center value of the pixel data of the model tooth in the channel in the multiple image frames is used as the reference pixel data of the model tooth in the pixel channel.

3. The method according to claim 1, characterized in that: The target image frame includes a plurality of frames, and the color of the tooth to be compared is determined based on the proximity between the pixel data of the tooth to be compared in the target image frame and the reference pixel data of the sample teeth of each color number, including: Determine a weight corresponding to each target image frame, wherein the weight is determined based on a shooting distance and / or a shooting angle of view of the tooth to be compared in the target image frame; Based on the weights, weighted average processing is performed on the pixel data of the teeth to be compared in the multiple target image frames to obtain fused pixel data; The degree of proximity between the fused pixel data and the reference pixel data of the sample tooth of each color number is determined, and the color number of the sample tooth with the greatest degree of proximity is used as the color number of the tooth to be compared.

4. The method according to claim 3, characterized in that The more the shooting distance of the to-be-colored tooth in the target image frame deviates from the specified distance range, the smaller the weight; and / or The more the tooth to be compared in color in the target image frame deviates from the specified shooting angle, the smaller the weight.

5. The method according to claim 1, characterized in that The target image frame includes a plurality of frames, and the color of the tooth to be compared is determined based on the proximity between the pixel data of the tooth to be compared in the target image frame and the reference pixel data of the sample teeth of each color number in the color comparison plate, including: For each target image frame, the color number of the sample tooth whose reference pixel data is closest to the pixel data of the tooth to be compared in the target image frame is used as the color number corresponding to the target image frame; The number of target image frames corresponding to each color number is counted, and the color number with the largest number is used as the color number of the tooth to be compared.

6. The method according to claim 1, characterized in that Determining the color of the tooth to be compared based on the pixel data of the tooth to be compared in the target image frame and the proximity to the reference pixel data of the sample teeth of each color number in the color comparison plate includes: Dividing the teeth to be compared in color in the target image frame into a plurality of tooth regions; For each tooth region, the color number of the sample tooth whose reference pixel data is closest to the pixel data of the tooth region is used as the color number of the tooth region; The color number of the tooth to be compared is determined based on the color numbers of the plurality of tooth regions.

7. The method according to claim 6, characterized in that The multiple tooth regions include a neck, a body, and an incisal end, and the color number of the tooth to be compared is determined based on the color numbers of the multiple tooth regions, including: The color number of the tooth to be compared is determined based on the weights corresponding to the multiple tooth regions and the color numbers of the multiple tooth regions, wherein the weight of the body is higher than the weights of the neck and the incisal end.

8. The method according to claim 1, characterized in that The pixel data of the tooth to be compared and the reference pixel data are both pixel data in the target color space. The degree of proximity between the pixel data of the tooth to be compared and the pixel data of the sample tooth of each color number in each target image frame is determined based on the following method: For each pixel channel in the target color space, determine the proximity between the pixel data of the tooth to be compared in the target image frame in the pixel channel and the reference pixel data of the sample tooth of the color number in the corresponding pixel channel; Based on the weight of each pixel channel, a weighted average processing is performed on the corresponding proximity of multiple pixel channels in the target color space to obtain the proximity between the pixel data of the tooth to be compared in the target image frame and the pixel data of the sample tooth of the color number, wherein the weight of each pixel channel is positively correlated with the sensitivity of the pixel value of the pixel channel to the change of the color number.

9. The method according to claim 1, characterized in that: The pixel data of the tooth to be compared in color and the reference pixel data are both pixel data in the LAB space.

10. The method according to claim 1, characterized in that After determining the color of the tooth to be compared, the method further includes: The color number of the tooth to be compared is displayed in the interactive interface, or a three-dimensional model of the tooth with the color is rendered.

11. The method according to claim 1, characterized in that: Determine the teeth to be shaded from a 3D model of the patient's mouth, including: The three-dimensional model is displayed in an interactive interface, and in response to the user's operation of selecting the tooth to be compared with the color, the tooth to be compared with the color is determined from the three-dimensional model of the patient's oral cavity; or The pre-trained AI model is used to identify the tooth to be repaired, and the tooth to be repaired or a tooth adjacent to the tooth to be repaired is used as the tooth to be compared in color.

12. A device for determining tooth color, characterized in that The device comprises: A module for determining teeth to be compared in color, used to determine teeth to be compared in color from a three-dimensional model of the patient's oral cavity, wherein the three-dimensional model is reconstructed based on a plurality of image frames collected by a three-dimensional scanning device for the patient's oral cavity; A selection module, for selecting a target image frame including the tooth to be compared in color from the multiple image frames; wherein the shooting distance of the tooth to be compared in the target image frame is within a preset distance range and / or the shooting angle of the tooth to be compared in the target image frame deviates from a specified shooting angle by less than a preset degree, and the specified shooting angle is the shooting angle of the three-dimensional scanning device facing the tooth to be compared in color; An acquisition module, used for acquiring reference pixel data of sample teeth of each color number in the color chart, wherein the reference pixel data of the sample teeth of each color number is determined based on an image frame of the sample teeth of the color number acquired by the three-dimensional scanning device; The colorimetric module is used to determine the color of the tooth to be compared based on the proximity between the pixel data of the tooth to be compared in the target image frame and the reference pixel data of the sample teeth of each color number.

13. An electronic device, characterized in that: The electronic device includes a processor, a memory, and computer instructions stored in the memory and executable by the processor. When the processor executes the computer instructions, the method according to any one of claims 1 to 11 is implemented.

Images