Oral cavity image data processing method and device, electronic equipment and readable storage medium

Abstract

The invention discloses an oral cavity image data processing method and device, electronic equipment and a readable storage medium, and relates to the technical field of computers, in particular to the technical fields of dental orthodontics, computer aided design and the like. According to the specific implementation scheme, based on two-dimensional images in at least two specified directions in an oral cavity, a tooth segmentation result of the two-dimensional image in each specified direction in the at least two specified directions and a tooth number corresponding to the tooth segmentation result are obtained; acquiring a three-dimensional tooth image with tooth number characteristics in the oral cavity; and correcting tooth numbers corresponding to the tooth segmentation results of the two-dimensional images in each specified direction by using the three-dimensional tooth images with tooth number characteristics in the oral cavity.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, in particular to the technical field of orthodontics, computer-aided design, etc. BACKGROUND

[0002] With the continuous development of computer science, the field of dentistry increasingly relies on computer technology. For example, in the field of orthodontics, in the entire process of digital orthodontic treatment, it is necessary to obtain intraoral photographs, i.e., two-dimensional images of teeth in the mouth, for instance-level segmentation, which can be applied to subsequent algorithms, such as disease recognition, bite recognition, etc. The existing technical solutions are usually for instance-level segmentation of teeth in intraoral photographs under a single viewing angle.

[0003] However, due to various factors such as algorithm uncertainty, differences in lighting conditions under different angles, etc., the tooth numbers in the instance-level segmentation of teeth in intraoral photographs under multiple viewing angles may not all be aligned, so that the same tooth of a patient is divided into two different tooth numbers by the algorithm in intraoral photographs under different viewing angles, which may affect the subsequent normal application.

[0004] Therefore, there is an urgent need to provide a method to obtain instance-level segmentation results of teeth in intraoral photographs with unified tooth numbers under different viewing angles, so as to improve the reliability of instance-level segmentation results of teeth in intraoral photographs. SUMMARY

[0005] The present disclosure provides a method and device for processing oral image data, an electronic device and a readable storage medium.

[0006] According to an aspect of the present disclosure, a method for processing oral image data is provided, comprising:

[0007] Based on two-dimensional images in at least two specified directions in the mouth, obtaining tooth segmentation results of two-dimensional images in each of the at least two specified directions and tooth numbers corresponding to the tooth segmentation results;

[0008] Obtaining a three-dimensional tooth image in the mouth with tooth number characteristics;

[0009] Using the three-dimensional tooth image in the mouth with tooth number characteristics, respectively correcting the tooth numbers corresponding to the tooth segmentation results of the two-dimensional images in each of the at least two specified directions.

[0010] According to another aspect of the present disclosure, a device for processing oral image data is provided, comprising:

[0011] an example segmentation unit configured to obtain a tooth segmentation result of a two-dimensional image in each of at least two specified directions in the oral cavity and a tooth number corresponding to the tooth segmentation result based on the two-dimensional image in each of the at least two specified directions in the oral cavity;

[0012] a feature acquisition unit configured to acquire a three-dimensional tooth image having a tooth number feature in the oral cavity;

[0013] a result correction unit configured to correct the tooth number corresponding to the tooth segmentation result of the two-dimensional image in each of the at least two specified directions in the oral cavity respectively based on the three-dimensional tooth image having the tooth number feature in the oral cavity.

[0014] According to yet another aspect of the present disclosure, an electronic device is provided, comprising:

[0015] at least one processor; and

[0016] a memory communicatively connected to the at least one processor; wherein

[0017] the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of the aspects and any possible implementation forms as described above.

[0018] According to still another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, the computer instructions being configured to cause a computer to perform the method of the aspects and any possible implementation forms as described above.

[0019] According to yet another aspect of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the method of the aspects and any possible implementation forms as described above.

[0020] According to the technical solution, the tooth segmentation result of the two-dimensional image in each of the at least two specified directions in the oral cavity and the tooth number corresponding to the tooth segmentation result are obtained based on the two-dimensional image in the at least two specified directions in the oral cavity, and then the three-dimensional tooth image with the tooth number feature in the oral cavity is obtained, so that the tooth number corresponding to the tooth segmentation result of the two-dimensional image in each of the at least two specified directions can be corrected by using the three-dimensional tooth image with the tooth number feature in the oral cavity. Since the tooth number corresponding to the tooth segmentation result of the two-dimensional image collected in different specified directions is corrected by using the three-dimensional tooth image and the tooth number feature, the tooth number corresponding to the tooth segmentation result of the two-dimensional image in each of the at least two specified directions after the correction can be consistent with the tooth number feature of the three-dimensional tooth image, and the tooth instance-level segmentation result of the two-dimensional image with the unified tooth number under different viewing angles in the oral cavity is obtained, thereby improving the reliability of the tooth instance-level segmentation result of the two-dimensional image collected in the oral cavity.

[0021] In addition, since the three-dimensional tooth image has less uncertainty in the tooth number feature, the tooth instance-level segmentation result of each two-dimensional image after the correction is more accurate, and the alignment of the tooth number in the tooth instance-level segmentation of the two-dimensional image under multiple viewing angles in the oral cavity can be effectively realized, thereby further improving the reliability of the tooth instance-level segmentation result of the two-dimensional image collected in the oral cavity.

[0022] In addition, by using the two-dimensional tooth image with the tooth number feature obtained by mapping and converting the three-dimensional tooth image with the tooth number feature in the oral cavity, the tooth number corresponding to the tooth segmentation result of the two-dimensional image collected in different specified directions is corrected, and the alignment of the tooth number in the tooth instance-level segmentation of the two-dimensional image under multiple viewing angles in the oral cavity can be more quickly realized, thereby further improving the efficiency of the tooth instance-level segmentation result of the two-dimensional image collected in the oral cavity.

[0023] In addition, the technical solution provided by the present disclosure can effectively improve the user experience.

[0024] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0025] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. The drawings are used to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0026] Figure 1 This is a schematic diagram based on the first embodiment of the present disclosure;

[0027] Figure 2 This is a schematic diagram according to the second embodiment of the present disclosure;

[0028] Figure 3 This is a block diagram of an electronic device used to implement the oral image data processing method of the embodiments of this disclosure. Detailed Implementation

[0029] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0030] Obviously, the described embodiments are only some, not all, of the embodiments disclosed herein. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0031] It should be noted that the terminals involved in the embodiments of this disclosure may include, but are not limited to, smart devices such as computers (PCs), mobile phones, personal digital assistants (PDAs), wireless handheld devices, and tablet computers; display devices may include, but are not limited to, personal computers, televisions, and other devices with display functions.

[0032] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0033] In the entire process of digital orthodontic treatment, instance-level segmentation of teeth from intraoral photographs is an essential step. Many subsequent algorithms, such as disease recognition and occlusion recognition, rely on the information from the preceding instance-level segmentation. Existing technical solutions typically focus on instance-level segmentation of teeth from intraoral photographs taken from a single viewpoint.

[0034] However, due to various factors such as algorithm uncertainty and differences in lighting conditions at different angles, the tooth numbers in the tooth instance-level segmentation of intraoral images from multiple perspectives may not be fully aligned. This results in the same tooth of a patient being divided into two different tooth numbers by the algorithm in intraoral images from different perspectives, which will affect subsequent normal applications.

[0035] To obtain accurate tooth instance-level segmentation results, tooth numbering optimization and alignment are required for intraoral photographs from multiple perspectives. Therefore, this application provides a method to obtain tooth instance-level segmentation results from intraoral photographs with uniform tooth numbering from different perspectives, thereby improving the reliability of tooth instance-level segmentation results from intraoral photographs.

[0036] Figure 1 This is a schematic diagram based on the first embodiment of the present disclosure, as shown below. Figure 1 As shown.

[0037] 101. Based on two-dimensional images of at least two specified directions within the oral cavity, obtain the tooth segmentation result of the two-dimensional image of each of the at least two specified directions and the tooth number corresponding to the tooth segmentation result.

[0038] The so-called designated direction can refer to the direction corresponding to the viewpoints such as up, down, left, right, and upright.

[0039] A two-dimensional image of the oral cavity in a specified direction can be image data acquired from directions corresponding to the top, bottom, left, right, and orthognathic views. Typically, one or more images can be acquired in each direction to serve as a two-dimensional image for that specified direction.

[0040] 102. Obtain a three-dimensional image of the teeth in the oral cavity that has tooth numbering features.

[0041] 103. Using the three-dimensional tooth images with tooth numbering features in the oral cavity, the tooth numbers corresponding to the tooth segmentation results of the two-dimensional images in each specified direction are corrected.

[0042] Specifically, the tooth numbering features in the three-dimensional tooth image can be used to identify the tooth number in the three-dimensional tooth image, and the tooth numbering features in the two-dimensional image can be used to identify the tooth number in the two-dimensional tooth image. Although they have different dimensions, both can be used to identify the tooth number in the corresponding tooth image.

[0043] Therefore, by using intraoral 3D tooth images and their tooth numbering features as reference standard data, the tooth numbers corresponding to the tooth segmentation results of 2D images acquired from different specified directions are uniformly corrected. This ensures that the tooth numbers corresponding to the tooth segmentation results of 2D images from each specified direction after correction are consistent with the tooth numbering features of the 3D tooth images, thus obtaining tooth instance-level segmentation results of 2D images with uniform tooth numbers from different intraoral perspectives.

[0044] In this application, the word "specify" in the specified application does not have any special meaning; it is simply used to specify the current operation object. Therefore, the specified direction is the ordinary direction corresponding to normal viewing angles such as up, down, left, right, and upright.

[0045] It should be noted that some or all of the execution entities of 101 to 103 can be applications located on the local terminal, or they can be functional units such as plug-ins or software development kits (SDKs) set in applications located on the local terminal, or they can be processing engines located on the network-side server, or they can be distributed systems located on the network side, such as processing engines or distributed systems in oral image data processing devices on the network side. This embodiment does not impose any particular limitations on these.

[0046] It is understood that the application may be a native program installed on the local terminal, or it may be a web application of a browser on the local terminal. This embodiment does not limit this.

[0047] In this way, by obtaining the tooth segmentation result and the corresponding tooth number of the two-dimensional image in each of the at least two specified directions based on two-dimensional images in the oral cavity, a three-dimensional tooth image with tooth numbering features in the oral cavity is obtained. This allows the tooth numbering corresponding to the tooth segmentation result of the two-dimensional image in each specified direction to be corrected using the three-dimensional tooth image with tooth numbering features in the oral cavity. Since the tooth numbering corresponding to the tooth segmentation result of the two-dimensional image acquired in different specified directions is corrected using the three-dimensional tooth image in the oral cavity and its tooth numbering features, the tooth numbering corresponding to the tooth segmentation result of the two-dimensional image in each specified direction after correction can be consistent with the tooth numbering features of the three-dimensional tooth image. This obtains the tooth instance-level segmentation result of the two-dimensional image with unified tooth numbering from different perspectives in the oral cavity, thereby improving the reliability of the tooth instance-level segmentation result of the two-dimensional image acquired in the oral cavity.

[0048] Optionally, in one possible implementation of this embodiment, in step 101, based on two-dimensional images of at least two specified directions within the oral cavity, specifically using existing algorithms, such as the YOLO neural network, the tooth segmentation result of the two-dimensional image of each specified direction in the at least two specified directions and the tooth number corresponding to the tooth segmentation result are obtained.

[0049] Typically, there are a total of 56 teeth in the oral cavity, including 32 permanent teeth, 20 deciduous teeth, and 4 supernumerary teeth.

[0050] At this point, due to various factors such as algorithm uncertainty and differences in lighting conditions at different angles, the tooth numbers corresponding to the tooth segmentation results of each specified direction 2D image obtained in this step may not be fully aligned. It is possible that the same tooth of the patient may be divided into two different tooth numbers by the algorithm in 2D images of different specified directions. Therefore, steps 102 and 103 need to be further executed to align the tooth numbers of multiple specified direction 2D images.

[0051] Optionally, in one possible implementation of this embodiment, in step 102, the two-dimensional image of each specified direction, the tooth segmentation result of the two-dimensional image of each specified direction, and the tooth number corresponding to the tooth segmentation result can be used to perform three-dimensional reconstruction processing to obtain the three-dimensional tooth image with tooth number features.

[0052] In this implementation, specific 3D reconstruction algorithms such as 3D Gaussian Splatting (GS), Neural Radiance Field (NeRF), Occupancy grid, Signed Distance Field (SDF), Plenoptic voxel (Plenoxel), and Instant Neural Graphics Primitives (Instant-NGP) can be used for 3D reconstruction processing.

[0053] In a specific implementation process, the parameter information of the acquisition device for the two-dimensional image in each specified direction can be obtained based on the two-dimensional image in each specified direction. Then, based on the two-dimensional image in each specified direction, the parameter information of the acquisition device for the two-dimensional image in each specified direction, the tooth segmentation result of the two-dimensional image in each specified direction, and the tooth number corresponding to the tooth segmentation result, three-dimensional reconstruction processing can be performed to obtain the three-dimensional tooth image with tooth number features.

[0054] In this implementation process, if parameter information of the acquisition device for the two-dimensional image in each specified direction exists, the parameter information of the acquisition device for the two-dimensional image in each specified direction can be directly obtained.

[0055] If the parameter information of the acquisition device for the two-dimensional image in each specified direction does not exist, the existing technical solution can be used to obtain the parameter information of the acquisition device for the two-dimensional image in each specified direction based on the two-dimensional image in each specified direction.

[0056] For example, a feature matching algorithm can be used to calculate the parameter information of the acquisition device for each specified direction of the two-dimensional image. For instance, COLMAP (COLLISION-MAPping) software can be used to calculate the parameter information of the acquisition device for a specific direction of the two-dimensional image. This involves using the Scale-Invariant Feature Transform (SIFT) algorithm to extract feature point data from the two-dimensional image, and then further calculating the parameter information of the acquisition device based on the extracted intraoral imaging feature point data using its built-in feature point matching process. Alternatively, a different feature matching algorithm can be used to extract the feature point data.

[0057] Alternatively, other methods such as in-situ calibration and neural networks can be used to obtain the parameter information of the acquisition device for the two-dimensional image in each specified direction.

[0058] For example, in 3D reconstruction, 3D GS optimizes the position, number, and feature vectors of a point cloud set. Specifically, each point in the point cloud is a colored multidimensional Gaussian function. In practice, the gradient descent algorithm can be implemented using the PyTorch framework to optimize the mean, covariance, spherical harmonic function (representing color), and transparency of the Gaussian function. During training, given the parameter information of the acquisition device for a 2D image in a specified orientation, the Gaussian point cloud is projected into 2D space for rendering. The rendered image is compared with the actual 2D image, and the parameters to be optimized in the Gaussian point cloud set are updated according to the set optimization rate and the loss function generated by the comparison. Through continuous iteration, the 3D features of the Gaussian point cloud can be consistent with the 3D tooth features corresponding to the 2D image in the specified orientation. In addition to traditional parameter optimization, the tooth number parameter is further optimized so that the reconstructed 3D tooth image has tooth numbering features. The actual annotation of the tooth number parameter can be done in various ways, such as obtaining the annotation result manually, or obtaining the annotation result by weighting the tooth numbers corresponding to the tooth segmentation results of multiple two-dimensional images in a specified direction.

[0059] The three-dimensional tooth image with tooth numbering features obtained after three-dimensional reconstruction processing may contain some errors. Therefore, the tooth numbering features can be further optimized.

[0060] For example, the tooth numbering features can be further optimized based on the physical characteristics of the teeth, such as the distance between the two tooth numbers from the middle position or the fact that the two tooth numbers are different, in order to adjust the tooth numbering in the three-dimensional tooth image.

[0061] Alternatively, for example, the Hungarian algorithm can be used to optimize the tooth numbering feature.

[0062] Because the tooth numbering features of the reconstructed 3D tooth images have been optimized, the obtained tooth numbers are all unique and reasonable.

[0063] Optionally, in one possible implementation of this embodiment, in step 102, the oral cavity may be scanned to obtain an original three-dimensional tooth image; based on the original three-dimensional tooth image, the three-dimensional tooth image with tooth numbering features may be obtained.

[0064] It is understood that the two-dimensional image obtained in the specified direction in this application may be an image acquired at the same time as the original three-dimensional tooth image obtained by scanning the oral cavity, or it may not necessarily correspond strictly to the original three-dimensional tooth image. It may be an image acquired after a period of time with a slight change in tooth position, such as a subtle change in teeth during the orthodontic stage. Subsequent feature matching and other processes have a certain degree of robustness.

[0065] Optionally, in one possible implementation of this embodiment, in step 103, the three-dimensional tooth image with tooth numbering features in the oral cavity can be mapped and transformed to obtain a two-dimensional tooth image with tooth numbering features. Then, the two-dimensional tooth image with tooth numbering features can be used to correct the tooth number corresponding to the tooth segmentation result of the two-dimensional image in each specified direction.

[0066] In a specific implementation process, a three-dimensional tooth image with tooth numbering features in the oral cavity can be projected to obtain a two-dimensional tooth image with tooth numbering features.

[0067] In another specific implementation, a three-dimensional image of teeth with tooth numbering features in the oral cavity can be rendered to obtain a two-dimensional image of teeth with tooth numbering features.

[0068] For example, by using ray tracing sampling, the tooth number corresponding to each pixel in the rendered 2D tooth image can be obtained, and the tooth number with the most pixels in the bounding box of each instance can be used as the tooth number of the corresponding instance.

[0069] In this embodiment, based on two-dimensional images from at least two specified directions within the oral cavity, the tooth segmentation result and the corresponding tooth number for each specified direction are obtained. This leads to the acquisition of a three-dimensional tooth image with tooth numbering features within the oral cavity. This allows for the correction of the tooth numbers corresponding to the tooth segmentation results of the two-dimensional images from each specified direction using the three-dimensional tooth image with tooth numbering features. Because the three-dimensional tooth image and its tooth numbering features are used to uniformly correct the tooth numbers corresponding to the tooth segmentation results of the two-dimensional images acquired from different specified directions, the tooth numbers corresponding to the tooth segmentation results of the two-dimensional images from each specified direction after correction are consistent with the tooth numbering features of the three-dimensional tooth image. This results in a tooth instance-level segmentation result of a two-dimensional image with uniform tooth numbers from different viewpoints within the oral cavity, thereby improving the reliability of the tooth instance-level segmentation result of the two-dimensional graphics acquired within the oral cavity.

[0070] Furthermore, by adopting the technical solution provided in this disclosure, the tooth numbering features of the three-dimensional tooth image have less uncertainty, making the tooth instance-level segmentation results of each corrected two-dimensional image more accurate. This effectively achieves full alignment of tooth numbers in the tooth instance-level segmentation of two-dimensional images from multiple perspectives within the oral cavity, thereby further improving the reliability of the tooth instance-level segmentation results of the two-dimensional graphics acquired within the oral cavity.

[0071] Furthermore, by employing the technical solution provided in this disclosure, and by using a two-dimensional tooth image with tooth numbering features obtained by mapping and transforming a three-dimensional tooth image with tooth numbering features in the oral cavity, the tooth numbers corresponding to the tooth segmentation results of two-dimensional images acquired from different specified directions can be corrected. This enables faster alignment of all tooth numbers in the tooth instance-level segmentation of two-dimensional images from multiple perspectives in the oral cavity, thereby further improving the efficiency of tooth instance-level segmentation results of two-dimensional graphics acquired in the oral cavity.

[0072] In addition, the technical solutions provided in this disclosure can effectively improve the user experience.

[0073] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this disclosure is not limited to the described order of actions, because according to this disclosure, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this disclosure.

[0074] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0075] Figure 2 This is a schematic diagram based on the second embodiment of the present disclosure, as shown below. Figure 2 As shown. The oral cavity image data processing device 200 of this embodiment may include an instance segmentation unit 201, a feature acquisition unit 202, and a result correction unit 203. The instance segmentation unit 201 is used to obtain a tooth segmentation result and the corresponding tooth number for each of the two-dimensional images in at least two specified directions within the oral cavity, based on the two-dimensional images in at least two specified directions. The feature acquisition unit 202 is used to acquire a three-dimensional tooth image with tooth number features within the oral cavity. The result correction unit 203 is used to correct the tooth number corresponding to the tooth segmentation result of each of the two-dimensional images in the specified directions using the three-dimensional tooth image with tooth number features within the oral cavity.

[0076] It should be noted that some or all of the oral image data processing device in this embodiment may be an application located on a local terminal, or it may be a plugin or software development kit (SDK) or other functional unit set in an application located on a local terminal, or it may be a processing engine located on a network-side server, or it may be a distributed system located on the network side, such as a processing engine or distributed system in a network-side oral image data processing platform, etc. This embodiment does not impose any particular limitations on this.

[0077] It is understood that the application may be a native program installed on the local terminal, or it may be a web application of a browser on the local terminal. This embodiment does not limit this.

[0078] Optionally, in one possible implementation of this embodiment, the feature acquisition unit 202 may be used to perform three-dimensional reconstruction processing using the two-dimensional image of each specified direction, the tooth segmentation result of the two-dimensional image of each specified direction, and the tooth number corresponding to the tooth segmentation result, to obtain the three-dimensional tooth image with tooth number features; or to perform scanning processing on the oral cavity to obtain the original three-dimensional tooth image; and to obtain the three-dimensional tooth image with tooth number features based on the original three-dimensional tooth image.

[0079] In a specific implementation, the feature acquisition unit 202 can be used to obtain parameter information of the acquisition device of the two-dimensional image in each specified direction based on the two-dimensional image in each specified direction; and to perform three-dimensional reconstruction processing based on the two-dimensional image in each specified direction, the parameter information of the acquisition device of the two-dimensional image in each specified direction, the tooth segmentation result of the two-dimensional image in each specified direction and the tooth number corresponding to the tooth segmentation result, so as to obtain the three-dimensional tooth image with tooth number feature.

[0080] Optionally, in one possible implementation of this embodiment, the result correction unit 203 may be used to perform mapping and transformation processing on the three-dimensional tooth image with tooth numbering features in the oral cavity to obtain a two-dimensional tooth image with tooth numbering features; and to use the two-dimensional tooth image with tooth numbering features to correct the tooth number corresponding to the tooth segmentation result of the two-dimensional image in each specified direction.

[0081] In a specific implementation process, the result correction unit 203 can be used to project a three-dimensional tooth image with tooth numbering features in the oral cavity to obtain a two-dimensional tooth image with tooth numbering features; or to render a three-dimensional tooth image with tooth numbering features in the oral cavity to obtain a two-dimensional tooth image with tooth numbering features.

[0082] It should be noted that, Figure 1 The method in the corresponding embodiment can be implemented by the oral cavity image data processing device provided in this embodiment. For a detailed description, please refer to... Figure 1 The relevant content in the corresponding embodiments will not be repeated here.

[0083] In this embodiment, the instance segmentation unit obtains the tooth segmentation result and the corresponding tooth number of the two-dimensional image in each of the at least two specified directions based on the two-dimensional images in the oral cavity. Then, the feature acquisition unit obtains the three-dimensional tooth image with tooth number features in the oral cavity. This allows the result correction unit to use the three-dimensional tooth image with tooth number features in the oral cavity to correct the tooth number corresponding to the tooth segmentation result of the two-dimensional image in each specified direction. Since the three-dimensional tooth image in the oral cavity and its tooth number features are used to correct the tooth number corresponding to the tooth segmentation result of the two-dimensional image acquired in different specified directions, the tooth number corresponding to the tooth segmentation result of the two-dimensional image in each specified direction after correction can be consistent with the tooth number features of the three-dimensional tooth image. This obtains the tooth instance-level segmentation result of the two-dimensional image with unified tooth number from different perspectives in the oral cavity, thereby improving the reliability of the tooth instance-level segmentation result of the two-dimensional image acquired in the oral cavity.

[0084] Furthermore, by adopting the technical solution provided in this disclosure, the tooth numbering features of the three-dimensional tooth image have less uncertainty, making the tooth instance-level segmentation results of each corrected two-dimensional image more accurate. This effectively achieves full alignment of tooth numbers in the tooth instance-level segmentation of two-dimensional images from multiple perspectives within the oral cavity, thereby further improving the reliability of the tooth instance-level segmentation results of the two-dimensional graphics acquired within the oral cavity.

[0085] Furthermore, by employing the technical solution provided in this disclosure, and by using a two-dimensional tooth image with tooth numbering features obtained by mapping and transforming a three-dimensional tooth image with tooth numbering features in the oral cavity, the tooth numbers corresponding to the tooth segmentation results of two-dimensional images acquired from different specified directions can be corrected. This enables faster alignment of all tooth numbers in the tooth instance-level segmentation of two-dimensional images from multiple perspectives in the oral cavity, thereby further improving the efficiency of tooth instance-level segmentation results of two-dimensional graphics acquired in the oral cavity.

[0086] In addition, the technical solutions provided in this disclosure can effectively improve the user experience.

[0087] The acquisition, storage, and application of the two-dimensional images and the original three-dimensional dental images involved in the technical solutions disclosed herein comply with the relevant laws and regulations and do not violate public order and good morals.

[0088] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0089] Figure 3A schematic block diagram of an example electronic device 300 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0090] like Figure 3 As shown, the electronic device 300 includes a computing unit 301, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 302 or a computer program loaded from a storage unit 308 into a random access memory (RAM) 303. The RAM 303 may also store various programs and data required for the operation of the electronic device 300. The computing unit 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0091] Multiple components in electronic device 300 are connected to I / O interface 305, including: input unit 306, such as keyboard, mouse, etc.; output unit 307, such as various types of displays, speakers, etc.; storage unit 308, such as disk, optical disk, etc.; and communication unit 309, such as network card, modem, wireless transceiver, etc. Communication unit 309 allows electronic device 300 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0092] The computing unit 301 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 301 performs the various methods and processes described above, such as methods for processing oral image data. For example, in some embodiments, the methods for processing oral image data can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 308. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 300 via ROM 302 and / or communication unit 309. When the computer program is loaded into RAM 303 and executed by the computing unit 301, one or more steps of the methods for processing oral image data described above can be performed. Alternatively, in other embodiments, the computing unit 301 can be configured to perform methods for processing oral image data by any other suitable means (e.g., by means of firmware).

[0093] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0094] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0095] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0096] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0097] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), the Internet, and blockchain networks.

[0098] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service ecosystem, addressing the shortcomings of traditional physical hosts and VPS (Virtual Private Server, or simply "VPS") services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0099] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0100] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for processing oral cavity image data, characterized in that, include: Based on two-dimensional images of at least two specified directions within the oral cavity, obtain the tooth segmentation result of the two-dimensional image of each of the at least two specified directions and the tooth number corresponding to the tooth segmentation result; Obtain three-dimensional images of teeth with tooth numbering features within the oral cavity; Using the three-dimensional tooth images with tooth numbering features in the oral cavity, the tooth numbers corresponding to the tooth segmentation results of the two-dimensional images in each specified direction are corrected.

2. The method according to claim 1, characterized in that, The process of acquiring a three-dimensional image of teeth with tooth numbering features within the oral cavity includes: Using the two-dimensional image of each specified direction, the tooth segmentation result of the two-dimensional image of each specified direction, and the tooth number corresponding to the tooth segmentation result, a three-dimensional reconstruction process is performed to obtain the three-dimensional tooth image with tooth number features; or The oral cavity is scanned to obtain an original three-dimensional tooth image; based on the original three-dimensional tooth image, a three-dimensional tooth image with tooth numbering features is obtained.

3. The method according to claim 2, characterized in that, The step of performing three-dimensional reconstruction processing using the two-dimensional image of each specified direction, the tooth segmentation result of the two-dimensional image of each specified direction, and the tooth number corresponding to the tooth segmentation result to obtain the three-dimensional tooth image with tooth number features includes: Based on the two-dimensional images in each specified direction, obtain the parameter information of the acquisition device for the two-dimensional images in each specified direction; Based on the two-dimensional image of each specified direction, the parameter information of the acquisition device of the two-dimensional image of each specified direction, the tooth segmentation result of the two-dimensional image of each specified direction, and the tooth number corresponding to the tooth segmentation result, a three-dimensional reconstruction process is performed to obtain the three-dimensional tooth image with tooth number features.

4. The method according to any one of claims 1-3, characterized in that, The step of using the three-dimensional tooth images with tooth numbering features within the oral cavity to correct the tooth numbers corresponding to the tooth segmentation results of the two-dimensional images in each specified direction includes: A mapping transformation process is performed on the three-dimensional tooth image with tooth numbering features in the oral cavity to obtain a two-dimensional tooth image with tooth numbering features. Using the two-dimensional tooth image with tooth numbering features, the tooth numbering corresponding to the tooth segmentation result of the two-dimensional image in each specified direction is corrected.

5. The method according to claim 4, characterized in that, The step of mapping and transforming the three-dimensional tooth image with tooth numbering features within the oral cavity to obtain a two-dimensional tooth image with tooth numbering features includes: The three-dimensional tooth image with tooth numbering features within the oral cavity is projected to obtain a two-dimensional tooth image with tooth numbering features; or The three-dimensional tooth image with tooth numbering features in the oral cavity is rendered to obtain a two-dimensional tooth image with tooth numbering features.

6. A device for processing oral cavity image data, characterized in that, include: An instance segmentation unit is used to obtain a tooth segmentation result and the tooth number corresponding to the two-dimensional image of each of the at least two specified directions in the oral cavity based on two-dimensional images of at least two specified directions. The feature acquisition unit is used to acquire three-dimensional images of teeth with tooth numbering features in the oral cavity; The result correction unit is used to correct the tooth numbers corresponding to the tooth segmentation results of the two-dimensional image in each specified direction using the three-dimensional tooth image with tooth numbering features in the oral cavity.

7. The apparatus according to claim 6, characterized in that, The feature acquisition unit is specifically used for Using the two-dimensional image of each specified direction, the tooth segmentation result of the two-dimensional image of each specified direction, and the tooth number corresponding to the tooth segmentation result, a three-dimensional reconstruction process is performed to obtain the three-dimensional tooth image with tooth number features. or The oral cavity is scanned to obtain an original three-dimensional tooth image; based on the original three-dimensional tooth image, a three-dimensional tooth image with tooth numbering features is obtained.

8. The apparatus according to claim 7, characterized in that, The feature acquisition unit is specifically used for Based on the two-dimensional images in each specified direction, obtain the parameter information of the acquisition device for each two-dimensional image in each specified direction; and Based on the two-dimensional image of each specified direction, the parameter information of the acquisition device of the two-dimensional image of each specified direction, the tooth segmentation result of the two-dimensional image of each specified direction, and the tooth number corresponding to the tooth segmentation result, a three-dimensional reconstruction process is performed to obtain the three-dimensional tooth image with tooth number features.

9. The apparatus according to any one of claims 6-8, characterized in that, The result correction unit is specifically used for A mapping transformation process is performed on the three-dimensional tooth image with tooth numbering features in the oral cavity to obtain a two-dimensional tooth image with tooth numbering features. as well as Using the two-dimensional tooth image with tooth numbering features, the tooth numbering corresponding to the tooth segmentation result of the two-dimensional image in each specified direction is corrected.

10. The apparatus according to claim 9, characterized in that, The result correction unit is specifically used for The three-dimensional tooth image with tooth numbering features in the oral cavity is projected to obtain a two-dimensional tooth image with tooth numbering features. or The three-dimensional tooth image with tooth numbering features in the oral cavity is rendered to obtain a two-dimensional tooth image with tooth numbering features.

11. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1-5.

12. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-5.

13. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method according to any one of claims 1-5.

Images