Method and apparatus for determining color of tooth, and device

Abstract

Provided in the embodiments of the present application are a method and apparatus for determining the color of a tooth, and a device. An image of a template tooth of each color number in a shade guide that is collected by a three-dimensional scanning device is used to determine reference pixel data of the template tooth of each color number; and after a tooth to be subjected to shade matching is selected from a reconstructed three-dimensional model of the oral cavity of a patient, a target image frame having a photographing distance and a photographing angle of view meeting certain conditions are selected from among a plurality of image frames which are collected by a three-dimensional scanner and are used for reconstructing the three-dimensional model; and on the basis of the degree of proximity between pixel data of the tooth to be subjected to shade matching in the target image frame and the pixel data of the template tooth of each color number, the color number of the tooth to be subjected to shade matching is determined.

Description

Methods, apparatus and equipment for determining tooth color Cross-references to related applications This application claims priority to Chinese Patent Application No. 202510387424.6, filed with the Chinese Patent Office on March 28, 2025, the entire contents of which are incorporated herein by reference. Technical Field

[0001] This application relates to the field of digital oral technology, and in particular to a method, apparatus, and device for determining tooth color. Background Technology

[0002] In dental practice, to ensure a high degree of color consistency between the restoration (such as porcelain crowns or veneers) and the patient's natural teeth during restoration, it is usually necessary to compare the patient's natural teeth with a sample tooth in a shade guide to determine the shade number of the patient's natural teeth. Some methods involve directly matching the natural teeth to the shade guide with the naked eye; however, the results are easily affected by factors such as ambient lighting conditions and the experience of the person comparing, resulting in inaccuracies and poor repeatability. Other methods use instruments such as spectrophotometers and colorimeters to match the natural tooth color, but the complexity of tooth shape leads to poor accuracy in the results. Therefore, it is necessary to provide a more accurate shade matching method. Summary of the Invention

[0003] This application provides a method, apparatus, and device for determining tooth color.

[0004] According to a first aspect of the embodiments of this application, a method for determining tooth color is provided. The method includes: determining a tooth to be compared from a three-dimensional model of a patient's oral cavity, wherein the three-dimensional model is reconstructed based on multiple image frames acquired by a three-dimensional scanning device for the patient's oral cavity; selecting a target image frame containing the tooth to be compared from the multiple image frames; wherein the shooting distance of the tooth to be compared in the target image frame is within a preset distance range and / or the deviation between the shooting angle of the tooth to be compared in the target image frame and a specified shooting angle is less than a preset degree, and the specified shooting angle is the shooting angle when the camera of the three-dimensional scanning device is facing the tooth to be compared; acquiring reference pixel data of sample teeth of each color number in a color comparison chart, wherein the reference pixel data of the sample teeth of each color number is determined based on the image frame of the sample teeth of that color number acquired by the three-dimensional scanning device; and determining the color of the tooth to be compared based on the closeness between the pixel data of the tooth to be compared in the target image frame and the reference pixel data of the sample teeth of each color number.

[0005] According to a second aspect of the embodiments of this application, an apparatus for determining tooth color is provided. The apparatus includes: a tooth determination module configured to determine the tooth to be compared from a three-dimensional model of a patient's oral cavity, wherein the three-dimensional model is reconstructed based on multiple image frames acquired by a three-dimensional scanning device for the patient's oral cavity; a selection module configured to select a target image frame containing the tooth to be compared from the multiple image frames; wherein the shooting distance of the tooth to be compared in the target image frame is within a preset distance range and / or the deviation between the shooting angle of the tooth to be compared in the target image frame and a specified shooting angle is less than a preset degree, and the specified shooting angle is the shooting angle of the three-dimensional scanning device facing the tooth to be compared; an acquisition module configured to acquire reference pixel data of sample teeth of each color number in a color matching chart, wherein the reference pixel data of the sample teeth of each color number is determined based on the image frame of the sample teeth of that color number acquired by the three-dimensional scanning device; and a color matching module configured to determine the color of the tooth to be compared based on the closeness between the pixel data of the tooth to be compared in the target image frame and the reference pixel data of the sample teeth of each color number.

[0006] According to a third aspect of the embodiments of this application, an electronic device is provided, including a processor, a memory, and computer instructions stored in the memory that are executable by the processor. When the processor executes the computer instructions, it can implement the method mentioned in the first aspect above.

[0007] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided, which stores computer instructions that, when executed, implement the method mentioned in the first aspect above.

[0008] In this embodiment, a 3D scanning device can be used to scan sample teeth of each color number in a color chart to acquire images of each sample tooth. Based on these images, the pixel data of each color number of the sample teeth can be determined as reference pixel data. After the user selects a tooth to be compared from the reconstructed 3D model of the patient's oral cavity, a target image frame with a shooting distance and shooting angle meeting certain conditions can be selected from multiple image frames acquired by the 3D scanner for reconstructing the 3D model. The color number of the tooth to be compared is determined based on the closeness of the pixel data of the tooth to be compared in the target image frame to the pixel data of each color number of the sample teeth. This embodiment can utilize the pose information and depth information of each image frame obtained during the 3D reconstruction process to select a target image frame with a shooting angle and shooting distance meeting certain conditions, thereby ensuring the imaging quality of the tooth to be compared in the target image frame and improving the accuracy and reliability of the color comparison results.

[0009] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the embodiments of this application. Attached Figure Description

[0010] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments conforming to the present application and, together with the specification, serve to explain the technical solutions of the embodiments of the present application.

[0011] Figure 1 is a schematic diagram of a colorimeter.

[0012] Figure 2 is a schematic diagram of a method for determining tooth color according to an embodiment of this application.

[0013] Figure 3 is a flowchart of a method for determining tooth color according to an embodiment of this application.

[0014] Figure 4 is a schematic diagram of an application scenario of one embodiment of this application.

[0015] Figure 5 is a schematic diagram of selecting a target image frame according to an embodiment of this application.

[0016] Figure 6 is a schematic diagram of dividing teeth into regions according to an embodiment of this application.

[0017] Figure 7 is a schematic diagram of the logical structure of a device for determining tooth color according to an embodiment of this application.

[0018] Figure 8 is a schematic diagram of the logical structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0019] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0020] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the embodiments of this application. The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items. Additionally, the term “at least one” herein means any combination of at least two of any one or more of a plurality.

[0021] It should be understood that although the terms first, second, third, etc., may be used to describe various information in the embodiments of this application, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of the embodiments of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0022] To enable those skilled in the art to better understand the technical solutions in the embodiments of this application, and to make the above-mentioned objectives, features and advantages of the embodiments of this application more apparent and understandable, the technical solutions in the embodiments of this application will be further described in detail below with reference to the accompanying drawings.

[0023] In dental care, when restoring a patient's teeth, in order to ensure that the color of the restoration (such as porcelain crowns, veneers, etc.) is highly consistent with the color of the patient's natural teeth and to improve the aesthetics of the restoration, it is usually necessary to compare the color of the patient's natural teeth with the sample teeth in the shade guide to determine the color number of the patient's natural teeth.

[0024] For example, as shown in Figure 1, this is a schematic diagram of a shade guide. The shade guide includes multiple sample teeth, each corresponding to a shade number. When determining the shade number of a patient's natural teeth, the patient's teeth can be matched with the colors of the sample teeth in the shade guide to determine the shade number of the patient's natural teeth. Subsequently, dental restorations can be made based on this shade number.

[0025] Traditional color matching methods are visual color matching, which involves directly comparing the color of a shade guide with the color of natural teeth. This method is easily affected by factors such as the dentist's experience, ambient lighting conditions, and eye strain, leading to unstable and poor repeatability results. With the development of digital technology, instrumental color matching methods are gradually being applied in clinical practice. Instruments such as colorimeters and spectrophotometers can reduce the interference of human factors to some extent. However, these instruments have limitations when measuring teeth with complex shapes. The measurement results are affected by the unevenness of the tooth surface and volume reflection, resulting in insufficient accuracy and reliability. Therefore, it is necessary to provide a more accurate tooth color matching scheme.

[0026] Based on this, this application provides a method for determining tooth color. Considering that digital oral impression technology is widely used in clinical practice, scanning the patient's oral cavity with a 3D scanning device can quickly and efficiently obtain a 3D model of the patient's oral cavity. To improve the accuracy and efficiency of scanning, 3D scanning devices typically employ advanced optical imaging technologies, such as blue light scanning or confocal microscopy. These imaging technologies can accurately extract the color information of the patient's teeth. Therefore, the applicant conceived of using a 3D scanning device to determine the color of the patient's teeth.

[0027] For example, as shown in Figure 2, a 3D scanning device can be used to scan sample teeth for each color number in a color chart to acquire images of each sample tooth. Based on these images, the pixel data of each color number of the sample teeth can be determined as reference pixel data. Furthermore, when performing 3D reconstruction of a patient's oral cavity, it is usually necessary to acquire multiple frames of images of the patient's oral cavity from different perspectives. Therefore, these images can be used to determine the pixel data of the tooth to be compared. Then, based on the closeness of the pixel data of the tooth to be compared to the reference pixel data of each color number of the sample teeth, the color number of the tooth to be compared can be determined. Further, considering that when using a 3D scanning device to scan a patient's oral cavity, the oral cavity environment is relatively complex due to the presence of the tongue, saliva, etc., the acquired images are easily affected by the oral environment, leading to unstable image quality. For example, when photographing the same tooth, the shooting angle and shooting distance can affect the image exposure, reflection, and color cast. Therefore, when determining the pixel data of the teeth to be compared based on images acquired by a 3D scanning device, a target image frame that meets certain conditions in terms of shooting angle and / or shooting distance can be selected to ensure the imaging quality of the selected target image frame and improve the accuracy of the color comparison results.

[0028] In some embodiments, the 3D scanning device that acquires images of each sample tooth and the 3D scanning device that scans the patient's oral cavity are the same scanning device. In some embodiments, during the 3D reconstruction process, the shooting angle and distance of the teeth in each frame can be determined by determining the depth information of each frame image and combining it with the angle of each frame image acquired by the 3D scanning device. Since the images required for color matching are exactly the images used by the 3D scanning device to reconstruct the 3D model of the oral cavity, the angle and / or distance information during the reconstruction process can be reused to select target image frames suitable for color matching.

[0029] Compared to ordinary cameras, which can only capture two-dimensional images but cannot directly know the distance of each point in the image from the lens (depth information), obtaining the depth and angle of each point in a two-dimensional image requires additional complex technology. In contrast, 3D scanning equipment captures three-dimensional point clouds or images with depth information. Its core operation involves using technologies such as structured light and laser scanning to calculate and record the three-dimensional coordinates (X, Y, Z) and spatial pose (position and angle) of each pixel in real time for subsequent 3D reconstruction. The pose and depth information obtained by 3D scanning equipment can be reused at zero cost for selecting target image frames.

[0030] The method for determining tooth color provided in this application embodiment can be executed by various electronic devices. For example, the electronic device can be a 3D scanning device or other devices that are communicatively connected to the 3D scanning device, such as mobile phones, tablets, laptops, cloud servers, etc. This application embodiment does not impose any limitations.

[0031] Figure 3 illustrates an application scenario of this application. In some embodiments, the method can be executed by another device that is communicatively connected to the 3D scanning device. This other device can have scanning software installed, allowing the user to set scanning parameters, etc. Simultaneously, the 3D scanning device can send the acquired images of the patient's oral cavity to the scanning software, enabling the software to reconstruct a 3D model of the patient's oral cavity and display it to the user. The scanning software can integrate a color matching function, allowing the user to select teeth for color matching from the reconstructed 3D model. The scanning software can then execute the method to determine the color number of the teeth for color matching and display it to the user.

[0032] Specifically, as shown in Figure 4, the method for determining tooth color provided in this application embodiment may include steps S402 to S408.

[0033] S402. Determine the teeth to be compared in color from a three-dimensional model of the patient's oral cavity, wherein the three-dimensional model is reconstructed based on multiple image frames acquired by a three-dimensional scanning device for the patient's oral cavity.

[0034] In step S402, a 3D scanning device can be used to scan the patient's oral cavity to acquire multiple image frames. Then, based on these multiple image frames, a 3D reconstruction of the patient's oral cavity can be performed to obtain a 3D model of the patient's oral cavity. When tooth color matching is required, the teeth to be matched can be determined from the 3D model of the patient's oral cavity.

[0035] The methods for determining the teeth to be compared in color from the patient's three-dimensional oral model can be varied, including manual determination by the user or automatic determination, and can be set according to actual needs.

[0036] For example, in some embodiments, the 3D model can be displayed in an interactive interface, and the tooth to be compared can be determined from the 3D model of the patient's oral cavity in response to the user's selection of the tooth to be compared. Considering that the tooth to be compared is typically a tooth that needs restoration, or a tooth adjacent to the tooth that needs restoration—for example, a tooth located to the left, right, above, or below the tooth that needs restoration—it is possible to display a 3D model of the patient's oral cavity to the user so that the user can determine the tooth to be compared based on that 3D model.

[0037] There are several ways for users to select a tooth for color comparison. For example, users can directly click or select a tooth in the 3D model displayed on the interactive interface as the tooth to be compared, or users can input the tooth position number of the tooth to be compared through the interactive component so that the tooth to be compared can be determined from the 3D model based on the tooth position number, or users can input voice commands so that the tooth to be compared can be determined based on the user's voice commands.

[0038] In some embodiments, a pre-trained AI model can be used to identify the tooth to be repaired from the aforementioned 3D model, and then the tooth to be repaired or the adjacent tooth to be repaired can be used as the tooth to be color-matched.

[0039] In some embodiments, users can also enter an identification command to log in to their personal account and then retrieve the user's relevant historical information, such as the patient's medical records recorded by the user, the information marked by the user in the 3D model, etc. Based on the historical information, the tooth to be restored is determined, and then the tooth to be restored or the adjacent teeth of the tooth to be restored are used as the teeth to be color-matched.

[0040] The 3D scanning device can be an oral scanner, an extraoral scanner, a facial scanner, an electronic colorimeter, or any other device that can reconstruct a 3D model of the patient's oral cavity based on the data collected by the device. This application does not impose any restrictions on such devices.

[0041] S404. Select a target image frame containing the tooth to be compared from the plurality of image frames; wherein, the shooting distance of the tooth to be compared in the target image frame is within a preset distance range and / or the deviation between the shooting angle of the tooth to be compared in the target image frame and the specified shooting angle is less than a preset degree, and the specified shooting angle is the shooting angle when the camera of the three-dimensional scanning device is facing the tooth to be compared.

[0042] In step S404, after determining the tooth to be compared, a target image frame containing the tooth can be selected from multiple image frames acquired by the 3D scanning device. However, considering that factors such as shooting angle and shooting distance affect the exposure and reflection areas of the image, resulting in a certain degree of color cast in the tooth, a target image frame with suitable shooting angle and shooting distance can be selected from the multiple image frames acquired by the 3D scanning device to ensure the imaging quality of the tooth in the target image frame. During the 3D reconstruction of the patient's oral cavity using images acquired by the 3D scanning device, the pose of each frame captured by the camera can be determined, and the depth information of each pixel in each frame can also be determined. Therefore, this information from the 3D reconstruction process can be reused to select a target image frame with suitable shooting angle and shooting distance from the multiple image frames acquired by the 3D scanning device. In some embodiments, the three-dimensional scanning device used to scan the patient's oral cavity and the three-dimensional scanning device used to scan the sample teeth for comparison of various shades in the color chart are the same three-dimensional scanning device. In other words, multiple image frames of the patient's oral cavity and image frames of the sample teeth are acquired by the same three-dimensional scanning device.

[0043] For example, as shown in Figure 5, after displaying a 3D model of the patient's oral cavity on the interactive interface, the user can select a tooth to be compared in the 3D model. Then, using this tooth as a reference, a target image frame that meets the shooting angle and shooting distance conditions is selected from multiple acquired image frames. The target image frame can be one or more frames. By setting the shooting angle and shooting distance as constraints, the target image frame that meets the requirements can be filtered from the video stream acquired by the 3D scanning device, ensuring the accuracy and consistency of the color comparison results.

[0044] The selected target image frame can meet one or more of the following conditions: the shooting distance of the tooth to be compared in the target image frame is within a preset distance range; the deviation between the shooting angle of the tooth to be compared in the target image frame and the specified shooting angle is less than a preset degree; wherein, the specified shooting angle is the shooting angle when the camera of the 3D scanning device is facing the tooth to be compared.

[0045] The preset distance range can be determined based on the hardware parameters of the 3D scanning device and experimental data obtained from multiple tests. For example, the preset distance range can be determined based on the depth of field range of the camera in the 3D scanning device to ensure clear imaging of the teeth to be compared. Alternatively, a suitable distance range can be determined through experimental data obtained from multiple tests, so that when the 3D scanning device takes pictures of the teeth to be compared within this distance range, the image quality of the teeth to be compared is high and the color distortion is small, etc.

[0046] Furthermore, considering that when a camera is directly facing an object, the captured image typically exhibits better image quality and is less prone to color distortion, the image quality tends to deteriorate as the shooting angle deviates from the angle directly facing the object. Therefore, when selecting a target image frame, the deviation between the shooting angle of the tooth to be compared and the specified shooting angle should be less than a preset threshold. This preset threshold can be determined based on the hardware parameters of the 3D scanning equipment and experimental data obtained from multiple trials, ensuring that the image quality of the tooth to be compared is better in target image frames where the deviation from the specified shooting angle is within the preset threshold.

[0047] S406. Obtain reference pixel data of sample teeth for each color number in the color chart, wherein the reference pixel data of sample teeth for each color number is determined based on the image frames of sample teeth for that color number acquired by the three-dimensional scanning device.

[0048] In step S406, reference pixel data for sample teeth of each color number in a predetermined color chart can be obtained. For example, for sample teeth of each color number in the color chart, a 3D scanning device can be used to scan the sample teeth to obtain one or more frames of images. Then, the pixel data of the sample teeth can be determined based on the one or more frames of images as reference pixel data. This reference pixel data includes the pixel values ​​of each pixel channel of the sample teeth in the target color space. For example, taking the target color space as the LAB color space, for each color number of sample teeth, the pixel values ​​of the sample teeth in the L channel, A channel, and B channel can be determined. The pixel values ​​of each channel can be the average or center value of the pixel values ​​of multiple pixels in the image region corresponding to the sample teeth in the image frame.

[0049] S408. Based on the pixel data of the tooth to be compared in the target image frame and the degree of similarity with the reference pixel data of the sample teeth of each color number, determine the color of the tooth to be compared.

[0050] In step S408, considering that if the colors of two objects are similar, then the pixel values ​​of the corresponding image regions of the two objects are also similar, after acquiring the target image frame and the pre-determined reference pixel data of sample teeth for each color number, the color of the tooth to be compared can be determined based on the similarity between the pixel data of the tooth to be compared in the target image frame and the reference pixel data of the sample teeth for each color number. After determining the color of the tooth to be compared, relevant prompts can be displayed on the interactive interface, such as displaying the color number corresponding to that color.

[0051] Considering that reference pixel data serves as a benchmark for comparison, and to ensure that the reference pixel data can more accurately and realistically reflect the color of the sample teeth under different acquisition environments, in some embodiments, when determining the reference pixel data for each color number of the sample teeth, a 3D scanning device can be used to scan the sample teeth of that color number from different angles and / or different distances, obtaining multiple image frames of the sample teeth. Thus, the acquired multiple image frames can cover various shooting angles and distances. For each pixel channel, the mean or center value of the pixel data of the sample teeth in that channel from these multiple image frames can be used as the reference pixel data of the sample teeth in that pixel channel. For example, as shown in the figure, taking the pixel data in the LAB color space as an example, for the L channel, since these multiple image frames cover different shooting angles and distances, the pixel values ​​of the L channel of the sample teeth in these multiple image frames can reflect the brightness distribution of the sample teeth under different shooting conditions. Therefore, the center or mean of these L channel pixel values ​​can be obtained as the reference pixel value of the sample teeth in the L channel. For other pixel channels, a similar method can be used to determine the reference pixel values ​​of the sample tooth in other pixel channels. This method eliminates scanning errors, and the determined reference data more accurately reflects the true pixel values ​​of the sample tooth.

[0052] In some embodiments, the selected target image frames include multiple frames. When determining the color of the tooth to be compared based on the proximity of the pixel data of the tooth to be compared in the target image frames to the reference pixel data of the sample teeth of each color number, the pixel data of the tooth to be compared in these multiple target image frames can be fused first to obtain fused pixel data. Then, the fused pixel data is compared with the reference pixel data of the sample teeth of each color number. Considering that the imaging quality and imaging effect of the tooth to be compared are different in different target image frames, in order to obtain pixel data that can accurately reflect the color information of the tooth to be compared, when fusing the pixel data of the tooth to be compared in multiple target image frames, the weight corresponding to each target image frame can be determined first. The weight corresponding to each target image frame can be determined based on the shooting distance and / or shooting angle of the tooth to be compared in that target image frame. Then, the pixel data of the tooth to be compared in multiple target image frames can be weighted and averaged based on the weight corresponding to each target image frame to obtain fused pixel data. Then, the fused pixel data can be compared with the reference pixel data of the sample teeth for each color number to obtain the degree of similarity between the fused pixel data and the reference pixel data of the sample teeth for each color number. The color number of the sample teeth with the highest degree of similarity is taken as the color number of the tooth to be compared.

[0053] In some embodiments, the greater the deviation of the shooting distance of the tooth to be compared in the target image frame from a specified distance range, the smaller the weight of the target image frame. This specified distance range can be a range set based on the hardware parameters of the 3D scanning device and empirical values. When the shooting distance is within this range, it can ensure that the tooth to be compared is clearly imaged, and the color of the tooth is less likely to be distorted.

[0054] In some embodiments, the greater the deviation of the tooth to be compared in the target image frame from the specified shooting angle, the smaller the weight of the target image frame. The specified shooting angle is the shooting angle from which the camera of the 3D scanning device is directly facing the tooth to be compared. Generally, when a camera is directly facing an object and taking a picture of that object, the image quality of that object in the captured image is better, and the colors are less likely to be distorted or off-color. Therefore, for each target image frame, the deviation between the shooting angle of the tooth to be compared in that target image frame and the angle from which the camera is directly facing the tooth can be determined; the greater the deviation, the smaller the weight of the target image frame.

[0055] In some embodiments, the selected target image frames include multiple frames. When determining the color of the tooth to be compared based on the proximity of the pixel data of the tooth to be compared in the target image frame to the reference pixel data of the sample teeth for each color number in the color chart, for each target image frame, the pixel data of the tooth to be compared in that target image frame can be compared with the reference pixel data of the sample teeth for each color number to determine the proximity. Then, the color number of the tooth to be compared with the highest proximity can be taken as the color number corresponding to that target image frame, thereby obtaining the color number corresponding to the tooth to be compared in each target image frame. Then, the number of target image frames corresponding to each color number can be counted, and the color number with the most occurrences can be taken as the color number of the tooth to be compared.

[0056] For example, suppose there are 5 target image frames selected, labeled as frame 1, frame 2, frame 3, frame 4, and frame 5. For each target image frame, the pixel data of the tooth to be compared in that frame is compared with the reference pixel data of the sample teeth for each color number in the color chart to determine the color number corresponding to the tooth to be compared in each target image frame. Assume the color numbers corresponding to the teeth to be compared in each target image frame are as follows: Frame 1: Color number A; Frame 2: Color number C; Frame 3: Color number B; Frame 4: Color number A; Frame 5: Color number A; Count the number of target image frames corresponding to each color number: Color number A: Frames 1, 4, and 5, a total of 3 frames; Color number B: Frame 3, a total of 1 frame; Color number C: Frame 2, a total of 1 frame;

[0057] Statistical results show that color number A corresponds to the most image frames, therefore the color of the tooth to be compared was finally determined to be color number A.

[0058] The above embodiments reduce the errors that may be caused by a single frame image by comparing and statistically analyzing multiple frames of images, thereby improving the accuracy and reliability of tooth color matching.

[0059] Considering that different areas of the same tooth may have different colors, in some embodiments, to more accurately determine the tooth color, when determining the color of the tooth to be compared based on the similarity between the pixel data of the tooth to be compared in the target image frame and the reference pixel data of the sample teeth for each color number in the shade guide, for each target image frame, the tooth to be compared in that target image frame can be divided into multiple tooth regions. For each tooth region, the pixel data of that tooth region can be compared with the reference data of the sample teeth for each color number to determine the similarity between the two. Then, the color number of the sample tooth with the highest similarity is taken as the color number of that tooth region. Then, the color number of the tooth to be compared can be determined based on the color numbers of each of these multiple tooth regions. For example, a tooth can be divided into a 9-grid, resulting in 9 regions. Through the above comparison process, it is determined that 5 of the 9 tooth regions have the color number A, 2 tooth regions have the color number B, and 2 tooth regions have the color number C. Therefore, the color number of the tooth can be determined as color number A.

[0060] In the above embodiments, by dividing the teeth into regions and then comparing them using these regions as units, the color of the teeth can be determined more accurately, which is especially suitable for scenarios where the color distribution of teeth is uneven.

[0061] In some embodiments, the multiple tooth regions include the neck, body, and incisal edge. When determining the shade number of the tooth to be compared based on the individual shade numbers of the multiple tooth regions, the shade number can be determined based on the weights of each tooth region and their respective shade numbers. The weight of the body is higher than that of the neck and incisal edge. As shown in Figure 6, a tooth can typically be divided into three parts: the incisal edge, body, and neck. The neck is located at the bottom of the tooth, in contact with the gingiva. Its enamel is thin, allowing the dentin color to show through, resulting in high color saturation. The body is the middle part of the tooth, which is usually more visible; therefore, clinical shade comparison often uses the body as the primary natural tooth color. The incisal edge is located at the top of the tooth, with thick enamel and thin dentin. The tooth color is primarily determined by the translucency and opalescence of the enamel. To achieve better aesthetic results in the prepared restoration, weights can be assigned to each tooth region based on its characteristics. The shade number of the tooth to be compared is then determined by combining the weights of each tooth region with the shade numbers of each region. Among these, considering that the body is located in the middle of the tooth and is easier to see, the weight of the body can be higher than that of the neck and incisal edge.

[0062] For example, assuming the target image consists of 5 frames, each frame can be divided into the neck, body, and incision edge, and the color number of each tooth region can be determined, as follows: Frame 1: Neck, color number A; Body, color number B; Incision edge, color number B Frame 2: Neck, color number A; Body, color number B; Incision edge, color number C Frame 3: Neck, color number B; Body, color number B; Incision edge, color number C Frame 4: Neck, color number A; Body, color number A; Incision edge, color number C Frame 5: Neck, color number A; Body, color number B; Incision edge, color number C Wherein, color number A: 4 times for neck, 1 time for body, 0 times for incision edge; Color number B: 1 time for neck, 4 times for body, 1 time for incision edge; Color number C: 0 times for neck, 0 times for body, 4 times for incision edge; Assuming the weight of the neck is 0.2, the weight of the body is 0.5, and the weight of the incisal edge is 0.3, the final scores for each shade are as follows: Shade A score: 4×0.2+0.5×1+0=1.3 Shade B score: 1×0.2+0.5×4+1×0.3=2.5 Shade C score: 4×0.3=1.2 Shade B has the highest score; therefore, the color of the tooth to be compared can be determined to be shade B.

[0063] In some embodiments, both the pixel data of the tooth to be compared and the reference pixel data are pixel data in the target color space. When determining the proximity between the pixel data of the tooth to be compared and the pixel data of the sample tooth for each color number in each target image frame, for any pixel channel in the target color space, the proximity between the pixel data of the tooth to be compared in that pixel channel in the target image frame and the reference pixel data of the sample tooth for that color number in the corresponding pixel channel can be determined. Then, based on the weight of each pixel channel, a weighted average is performed on the proximity corresponding to each of the multiple pixel channels in the target color space to obtain the final proximity. The weight of different pixel channels is determined based on the sensitivity of the pixel value of that pixel channel to changes in color number; the higher the sensitivity, the greater the weight of the pixel channel. The target color space can be any color space, such as LAB, RGB, HSV, etc. Considering that different pixel channels have varying sensitivities to color changes, for example, taking LAB as the target color space, for sample teeth with color numbers B1, B2, and B3, as the color number changes, the pixel values ​​of the L and A channels in the LAB color space do not change significantly, but the pixel value of the B channel will continuously increase. That is, color number changes have a greater impact on the pixel value of the B channel. To better distinguish different color numbers and accurately determine the color of the teeth to be compared, the weights of pixel values ​​in different pixel channels can be set differently when comparing pixel data. The weights of pixel channels sensitive to color changes can be set higher.

[0064] In some embodiments, both the pixel data of the tooth to be compared and the reference pixel data are pixel data in the LAB color space. The LAB color space is a color model based on human visual perception, capable of more accurately reflecting the three dimensions of color: lightness (L), chroma (a, b). Compared to the traditional RGB color space, the LAB color space is less dependent on lighting conditions (i.e., less affected by lighting conditions), and can more accurately measure and describe tooth color. By comparing the pixel data of the tooth to be compared with the reference pixel data of sample teeth for each color number in the LAB color space, the accuracy and stability of color matching can be improved, resulting in more accurate color matching results.

[0065] In some embodiments, after determining the color of the tooth to be compared, the color code of the tooth can also be displayed in the interactive interface. For example, the color code can be displayed near the tooth in the interactive interface. For instance, taking this method as an example executed by scanning software, the user can move the cursor to different teeth in the 3D model. When the cursor stops on a tooth, the scanning software can determine the color code of that tooth using the above method, and then display the color code near that tooth. For example, when the cursor stops on a tooth, the scanning software determines the tooth position number and the tooth color code, displays the tooth position number and color code simultaneously in a certain area of ​​the screen, generates a corresponding table, or retrieves and displays an image of that color code from the color matching chart.

[0066] In some embodiments, to allow users to see the effect of the color on teeth more intuitively, a 3D model of teeth rendered with that color can also be directly displayed on the interactive interface.

[0067] In some embodiments, when determining the color of a tooth to be compared based on the proximity of the pixel data of the tooth to be compared in the target image frame to the reference pixel data of the sample teeth of each color number, the three color numbers with the highest proximity of the reference pixel data to the pixel data of the tooth to be compared in the target image frame can be determined, and then these three color numbers can be displayed in the interactive interface.

[0068] Considering the low resolution and poor imaging quality of 3D scanning equipment, this invention selects images with better imaging quality based on the shooting angle and distance from the input raw data. Furthermore, since 3D scanning equipment can determine pose and depth information during 3D reconstruction, this pose and depth information can be reused to determine the shooting angle and distance, and different subsequent processing procedures can be used for pixel-based data comparison. Moreover, when performing color matching based on the proximity of pixel data, this application can adjust the weight of pixel data in each pixel channel based on the sensitivity of different pixel channels to color codes, thereby improving the distinguishability of different color codes and thus enhancing the accuracy of the color matching results.

[0069] The tooth color matching scheme provided in this application embodiment can achieve fully automated processing, significantly improving operational convenience compared to traditional manual color matching methods such as endoscopy. Furthermore, this scheme can determine the reference pixel data of the sample tooth and the pixel data of the tooth to be matched based on multiple frames of images acquired from different angles or shooting distances using a 3D scanning device. By fusing the scanning results from multiple perspectives, it can provide more stable and reliable color matching results. Moreover, compared to schemes based on AI models, this scheme does not require a large amount of training data, especially for data on rare tooth colors. Furthermore, this scheme has high transferability and can be quickly transferred to various different color matching charts.

[0070] It is easy to understand that the solutions described in the above embodiments can be freely combined to obtain new solutions when there is no conflict. Due to space limitations, each solution is not listed in the embodiments of this application.

[0071] Accordingly, this application also provides a device for determining tooth color, as shown in FIG7. The device 70 includes: a tooth color determination module 71, configured to determine the tooth color to be compared from a three-dimensional model of the patient's oral cavity, wherein the three-dimensional model is reconstructed based on multiple image frames acquired by a three-dimensional scanning device for the patient's oral cavity; and a selection module 72, configured to select a target image frame containing the tooth color to be compared from the multiple image frames; wherein the shooting distance of the tooth color to be compared in the target image frame is within a preset distance range and / or the shooting distance of the tooth color to be compared in the target image frame is within a preset distance range. The deviation between the viewing angle and the specified shooting angle is less than a preset degree. The specified shooting angle is the shooting angle from which the 3D scanning device is facing the tooth to be compared. The acquisition module 73 is configured to acquire reference pixel data of sample teeth of each color number in the color matching chart. The reference pixel data of the sample teeth of each color number is determined based on the image frame of the sample teeth of that color number acquired by the 3D scanning device. The color matching module 74 is configured to determine the color of the tooth to be compared based on the closeness between the pixel data of the tooth to be compared in the target image frame and the reference pixel data of the sample teeth of each color number.

[0072] The specific steps of the task processing method performed by the above-mentioned device can be referred to the description in the above method embodiments, and will not be repeated here.

[0073] Furthermore, this application embodiment also provides an electronic device, as shown in FIG8, the device including a processor 81, a memory 82, and computer instructions stored in the memory 82 that can be executed by the processor 81, wherein the processor 81 executes the computer instructions to implement the method described in any of the above embodiments.

[0074] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the methods described in any of the foregoing embodiments.

[0075] Computer-readable media include both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0076] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that the embodiments of this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions of the embodiments of this application, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM (Read-only Memory) / RAM (Random access memory), magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0077] The systems, devices, modules, or units described in the above embodiments can be implemented by a computer processor or entity, or by a product with a certain function. A typical implementation device is a computer, which can be a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0078] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on its differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate. When implementing the embodiments of this application, the functions of each module can be implemented in one or more software and / or hardware. Alternatively, some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0079] The above descriptions are some implementation methods of the embodiments of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the embodiments of this application, and these improvements and modifications should also be considered as the protection scope of the embodiments of this application. Industrial applicability

[0080] The technical solution of this application embodiment uses a 3D scanning device to scan sample teeth of each color number in a color chart to acquire images of each sample tooth. Based on these images, the pixel data of each color number of the sample teeth is determined as reference pixel data. When a user selects a tooth to be compared from the reconstructed 3D model of the patient's oral cavity, a target image frame with a shooting distance and shooting angle meeting certain conditions can be selected from multiple image frames acquired by the 3D scanner used to reconstruct the 3D model. The color number of the tooth to be compared is determined based on the closeness of the pixel data of the tooth to be compared in the target image frame to the pixel data of each color number of the sample teeth. This application embodiment can utilize the pose information and depth information of each image frame obtained during the 3D reconstruction process to select a target image frame with a shooting angle and shooting distance meeting certain conditions, thereby ensuring the imaging quality of the tooth to be compared in the target image frame and improving the accuracy and reliability of the color comparison results.

Claims

1. A method for determining tooth color, comprising: The teeth to be color matched are determined from a three-dimensional model of the patient's oral cavity, wherein the three-dimensional model is reconstructed based on multiple image frames acquired by a three-dimensional scanning device for the patient's oral cavity; Select a target image frame containing the tooth to be compared from the plurality of image frames; wherein, the shooting distance of the tooth to be compared in the target image frame is within a preset distance range and / or the deviation between the shooting angle of the tooth to be compared in the target image frame and the specified shooting angle is less than a preset degree, wherein the specified shooting angle is the shooting angle when the camera of the three-dimensional scanning device is facing the tooth to be compared. The reference pixel data of the sample teeth for each color number in the color chart is obtained, wherein the reference pixel data of the sample teeth for each color number is determined based on the image frames of the sample teeth for that color number acquired by the three-dimensional scanning device. The color of the tooth to be compared is determined based on the pixel data of the tooth to be compared in the target image frame and the similarity of the reference pixel data of the sample teeth of each color number.

2. The method according to claim 1, wherein, The reference pixel data for the sample teeth of each color number is determined based on the following method: The three-dimensional scanning device is used to scan the sample teeth of the color number from different angles and / or different distances to obtain multiple image frames of the sample teeth; For each pixel channel, the mean or center value of the pixel data of the template tooth in that channel in the multiple image frames is used as the reference pixel data of the template tooth in that pixel channel.

3. The method according to claim 1 or 2, wherein, The target image frames include multiple frames. Based on the pixel data of the tooth to be compared in the target image frames and the closeness of the pixel data of the sample teeth for each color number, the color of the tooth to be compared is determined, including: Determine the weight corresponding to each target image frame, the weight being determined based on the shooting distance and / or shooting angle of the tooth to be compared in that target image frame; Based on the weights, the pixel data of the teeth to be color matched in the multiple target image frames are weighted and averaged to obtain fused pixel data; Determine the degree of similarity between the fused pixel data and the reference pixel data of the sample teeth for each color number, and take the color number of the sample teeth with the greatest similarity as the color number of the teeth to be compared.

4. The method according to claim 3, wherein, The greater the deviation of the shooting distance of the tooth to be compared in the target image frame from the specified distance range, the smaller the weight; and / or The more the teeth to be compared deviate from the specified shooting angle in the target image frame, the smaller the weight.

5. The method according to any one of claims 1 to 4, wherein, The target image frames include multiple frames. Based on the pixel data of the tooth to be compared in the target image frames and the closeness of the pixel data to the reference pixel data of sample teeth for each color number in the color chart, the color of the tooth to be compared is determined, including: For each target image frame, the color number of the sample tooth whose reference pixel data is closest to the pixel data of the tooth to be compared in that target image frame is taken as the color number of that target image frame. The number of target image frames corresponding to each color number is counted, and the color number with the most counts is taken as the color number of the tooth to be compared.

6. The method according to any one of claims 1 to 5, wherein, Based on the pixel data of the tooth to be compared in the target image frame, and the degree of similarity between this pixel data and the reference pixel data of the sample teeth for each color number in the color chart, the color of the tooth to be compared is determined, including: The teeth to be compared in the target image frame are divided into multiple tooth regions; For each tooth region, the color number of the sample tooth whose reference pixel data is closest to the pixel data of that tooth region is taken as the color number of that tooth region. The color number of the tooth to be compared is determined based on the color number of each of the multiple tooth regions.

7. The method according to claim 6, wherein, The plurality of tooth regions include the neck, body, and incisal edge. The color number of the tooth to be compared is determined based on the color number of each of the plurality of tooth regions, including: The color number of the tooth to be compared is determined based on the weights of each of the multiple tooth regions and the color numbers of each of the multiple tooth regions, wherein the weight of the body is higher than that of the neck and the incisal edge.

8. The method according to any one of claims 1 to 7, wherein, The pixel data of the tooth to be compared and the reference pixel data are both pixel data in the target color space. The degree of similarity between the pixel data of the tooth to be compared and the pixel data of the sample tooth for each color number in each target image frame is determined based on the following method: For each pixel channel in the target color space, determine the pixel data of the tooth to be compared in the target image frame in that pixel channel and the degree of similarity between it and the reference pixel data of the sample tooth of that color number in the corresponding pixel channel. Based on the weight of each pixel channel, a weighted average is performed on the similarity of the corresponding pixel channels in the target color space to obtain the similarity between the pixel data of the tooth to be compared and the pixel data of the sample tooth of the same color in the target image frame. The weight of each pixel channel is positively correlated with the sensitivity of the pixel value of that pixel channel to changes in color.

9. The method according to any one of claims 1 to 8, wherein, The pixel data of the tooth to be compared and the reference pixel data are both pixel data in LAB space.

10. The method according to any one of claims 1 to 9, wherein, After determining the color of the tooth to be compared, the method further includes: The interactive interface displays the color number of the tooth to be compared or renders a 3D model of the tooth with that color.

11. The method according to any one of claims 1 to 10, wherein, The teeth to be compared for color are determined from a three-dimensional model of the patient's oral cavity, including: The three-dimensional model is displayed in the interactive interface, and in response to the user's selection of the tooth to be compared, the tooth to be compared is determined from the three-dimensional model of the patient's oral cavity.

12. The method according to any one of claims 1 to 10, wherein, The teeth to be compared for color are determined from a three-dimensional model of the patient's oral cavity, including: A pre-trained AI model is used to identify the tooth to be repaired, and the tooth to be repaired or the adjacent tooth is used as the tooth to be color-matched.

13. An apparatus for determining tooth color, comprising: The tooth to be matched determination module is configured to determine the tooth to be matched from a three-dimensional model of the patient's oral cavity, wherein the three-dimensional model is reconstructed based on multiple image frames acquired by a three-dimensional scanning device for the patient's oral cavity; The selection module is configured to select a target image frame containing the tooth to be compared from the plurality of image frames; wherein the shooting distance of the tooth to be compared in the target image frame is within a preset distance range and / or the deviation between the shooting angle of the tooth to be compared in the target image frame and a specified shooting angle is less than a preset degree, and the specified shooting angle is the shooting angle of the three-dimensional scanning device facing the tooth to be compared. The acquisition module is configured to acquire reference pixel data of sample teeth for each color number in the color chart, wherein the reference pixel data of sample teeth for each color number is determined based on image frames of sample teeth for that color number acquired by the three-dimensional scanning device. The color matching module is configured to determine the color of the tooth to be matched based on the proximity of the pixel data of the tooth to be matched in the target image frame to the reference pixel data of the sample teeth of each color number.

14. An electronic device comprising a processor, a memory, and computer instructions stored in the memory executable by the processor, wherein the processor executes the computer instructions to implement the method as claimed in any one of claims 1-12.

Images