Dental jaw recognition and display method and apparatus, oral instrument, storage medium, and device

Abstract

Disclosed in the present invention are a dental jaw recognition and display method and apparatus, an oral instrument, a storage medium, and a device. The dental jaw recognition method comprises: obtaining first dental jaw model data and second dental jaw model data of teeth to be detected; obtaining relationship identification information of tooth pairs of the teeth to be detected according to the first dental jaw model data and the second dental jaw model data; and determining dental jaw information corresponding to the first dental jaw model data and the second dental jaw model data according to the relationship identification information. According to the dental jaw recognition method provided by the present invention, the robustness of the dental jaw recognition process can be higher, and excessive manual intervention and complex pretreatment steps are avoided. The method features both high efficiency and stability and can adapt to various complex scenarios.

Description

A method, device, oral appliance, storage medium and equipment for tooth and jaw identification and display

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of the People's Republic of China on December 31, 2023, with application number 202311872467.0 and application name “A method, device, oral instrument, storage medium and equipment for dental identification and display”, the entire contents of which are incorporated by reference into this application; this application claims priority to the Chinese patent application filed with the State Intellectual Property Office of the People's Republic of China on August 27, 2024, with application number 202411190095.8 and application name “A method and device for occlusion adjustment”, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present invention relates to the technical field of oral digital models, and in particular to a method, device, oral appliance, storage medium and equipment for tooth and jaw identification and display. Background Art

[0004] With the development of society and the increasing demand for quality of life, maintaining oral health, including oral hygiene and aesthetics, has gradually become a focus of attention. When analyzing oral structures, including crowns and gums, it is inevitable to use methods such as oral scanning, intraoral photography, and silicone model digitization to construct a digital model of the oral cavity. This digital model is then analyzed to determine the ultimate treatment plan.

[0005] When performing model analysis, it is necessary to distinguish between the maxillary and mandibular parts of the model. In the general technical field, the jaws are mainly marked manually to achieve the effect of distinction, but this method has low efficiency and accuracy, and cannot cope with complex and variable individual differences. The prior art provides a method for segmenting the alveolar bone, in which a model containing only the alveolar bone is input into the lower alveolar bone segmentation model and the upper alveolar bone segmentation model in turn, but this technical solution has strict requirements on the data preprocessing step, and will cause the error to continue to increase during the iteration process, and the segmentation effect is still not ideal. Summary of the Invention

[0006] One of the purposes of the present invention is to provide a method for tooth and jaw recognition to solve the technical problems in the prior art of over-reliance on manual marking and preprocessing steps, as well as the single calculation reference object leading to increasing errors during the iterative process.

[0007] In a first aspect, an embodiment of the present application provides a method for tooth and jaw recognition, comprising:

[0008] respectively obtaining first and second jaw model data of the tooth to be tested, wherein the first and second jaw models correspond to different jaws;

[0009] Obtaining, based on the first jaw model data and the second jaw model data, relationship identification information of a tooth pair to be measured, wherein the tooth pair includes at least one pair of teeth located in different jaws and corresponding to each other, and the relationship identification information is used to indicate the relative positional relationship of the tooth pair;

[0010] According to the relationship identification information, the dental information corresponding to the first dental model data and the second dental model data is determined.

[0011] Compared with the existing technology, the jaw recognition method provided by the present invention uses the relational identification information that characterizes the relative relationship of the tooth pairs to determine the jaw information of different jaws where the tooth pairs are located, which is equivalent to providing the tooth pairs as a reference for the jaw recognition process, making the overall recognition process more robust; and, since the tooth recognition scheme is relatively mature, compared with the scheme with excessive manual intervention and complex preprocessing steps, it takes into account the characteristics of high efficiency and stability; since the tooth pairs are discrete, it can also avoid individual errors affecting the overall results, and can adapt to recognition in a variety of complex scenarios.

[0012] Optionally, the dental information includes maxillary determination information or mandibular determination information for the first dental model, and maxillary determination information or mandibular determination information for the second dental model.

[0013] Optionally, obtaining the relationship identification information of the tooth pair to be measured based on the first jaw model data and the second jaw model data includes:

[0014] Based on the first jaw model data and the second jaw model data, obtaining tooth pair data of at least one pair of teeth located in different jaws and corresponding to each other;

[0015] The tooth pair data are input in parallel into a tooth-jaw relationship recognition model to predict corresponding relationship identification information; the tooth-jaw relationship recognition model is a neural network model, which is trained based on preset tooth pair data and relationship labels that characterize the relative positional relationship of the tooth pairs in the preset tooth pair data.

[0016] Optionally, the step of inputting the tooth pair data into a tooth-jaw relationship recognition model in parallel to predict corresponding relationship identification information includes:

[0017] Perform feature extraction on the tooth data to be tested according to the first feature extraction module to obtain first tooth data, and perform feature extraction on the tooth data to be tested according to the second feature extraction module to obtain second tooth data;

[0018] obtaining first feature data and second feature data corresponding to the first tooth data and the second tooth data, respectively, and generating a tooth pair feature sequence;

[0019] Relationship identification information between the first tooth and the second tooth is predicted based on the tooth pair feature sequence.

[0020] Optionally, determining the dental information corresponding to the first dental model data and the second dental model data according to the relationship identification information includes:

[0021] The numerical form of the relationship identification information is obtained, and the upper and lower jaw determination information of the first jaw model data and the second jaw model data is determined according to the numerical relationship.

[0022] Optionally, the numerical form of the relationship identification information includes:

[0023] When, in the tooth pair data, the tooth to be measured located in the first jaw model points to the maxillary tooth, and the tooth to be measured located in the second jaw model data points to the mandibular tooth, relationship identification information in a numerical form of a first label value is obtained;

[0024] In the tooth pair data, when the tooth to be measured in the first jaw model points to the mandibular tooth and the tooth to be measured in the second jaw model data points to the maxillary tooth, relationship identification information in a numerical form of a second label value is obtained.

[0025] Optionally, determining the upper and lower jaw determination information of the first jaw model data and the second jaw model data according to the numerical relationship includes:

[0026] The upper and lower jaw determination information is determined based on the numerical label values ​​corresponding to the plurality of tooth pairs and according to the voting score results.

[0027] Optionally, any tooth pair data is obtained by:

[0028] dividing the first jaw model and the second jaw model to obtain a plurality of first jaw tooth data and a plurality of second jaw tooth data;

[0029] The tooth data of the first jaw and the tooth data of the second jaw are matched according to the tooth position numbers to obtain tooth pair data.

[0030] Optionally, dividing the first dental model and the second dental model includes:

[0031] According to the degree of differentiation of the parts in the dental models, the surfaces of the first and second dental models are simplified;

[0032] The first and second jaw models are registered respectively, and the first and second jaw models are segmented based on a tooth segmentation model, wherein the tooth segmentation model is a neural network model trained based on a preset jaw model and preset tooth labels representing tooth distribution in the preset jaw model.

[0033] Optionally, the surface simplification of the first and second dental models according to the degree of differentiation of the parts in the dental models includes:

[0034] According to the degree of distinction between adjacent facets in the dental model, the facet segmentation method of the dental model is determined, and the facets of the dental model are obtained accordingly;

[0035] Merge the valid vertices of the obtained patch.

[0036] Optionally, determining the face segmentation method of the dental model according to the degree of distinction between adjacent facets in the dental model includes:

[0037] The face segmentation method of the dental model is determined based on the degree of distinction between the face and other intraoral tissues and the cost of adjacent face patches belonging to the same intraoral tissue; the degree of distinction between the face and other intraoral tissues is determined based on at least one of the following indicators:

[0038] How close the front piece is to the line connecting the incisal ends;

[0039] The degree to which the front slice approaches the incisal extreme point of the corresponding tooth;

[0040] The degree to which the current front film is away from the model center of the dental model data.

[0041] In a second aspect, an embodiment of the present application provides a display method, including:

[0042] Displaying a first jaw model and a second jaw model according to the jaw information, wherein the jaw information is obtained by executing any of the jaw recognition methods described in the first aspect above;

[0043] The first dental model and the second dental model are used to design a correction plan.

[0044] In a third aspect, an embodiment of the present application provides an oral appliance, which is prepared based on the tooth and jaw information determined by any tooth and jaw recognition method described in the first aspect.

[0045] In a fourth aspect, an embodiment of the present application provides a computer storage medium on which an application is stored. When the application is executed, the steps of the tooth and jaw recognition method described in any of the first aspects are implemented.

[0046] In a fifth aspect, an embodiment of the present application provides a tooth and jaw identification device, comprising:

[0047] The first module is used to obtain first and second jaw model data of the tooth to be tested; the first and second jaw models correspond to different jaws;

[0048] a second module, configured to obtain, based on the first jaw model data and the second jaw model data, relationship identification information of a tooth pair to be measured, wherein the tooth pair includes at least one pair of teeth located in different jaws and corresponding to each other, and the relationship identification information is used to indicate the relative positional relationship of the tooth pair;

[0049] The third module is used to determine the dental information corresponding to the first dental model data and the second dental model data according to the relationship identification information.

[0050] In a sixth aspect, an embodiment of the present application provides an electronic device, comprising a processor, a memory, and a communication bus, wherein the processor and the memory communicate with each other via the communication bus;

[0051] The memory is used to store application programs;

[0052] The processor is configured to implement any of the steps of the tooth and jaw recognition method described in the first aspect when executing the application stored in the memory.

[0053] In a seventh aspect, embodiments of the present specification provide an occlusion adjustment method, comprising:

[0054] Obtaining a first digital model and a second digital model of the user's oral cavity, wherein the first digital model represents a digital model obtained by digitizing the user's oral cavity, and the second digital model represents a digital model obtained by collecting oral data of the user's oral cavity in an occluded state;

[0055] Performing model registration on the first digital model and the second digital model to obtain a spatial transformation relationship for mapping the first digital model to the second digital model;

[0056] The occlusal relationship of the first digital model is adjusted based on the spatial conversion relationship to obtain a target digital model of the user's oral cavity.

[0057] In some embodiments, the first digital model and the second digital model are digital models of the dental area of ​​the user's mouth, and obtaining the first digital model and the second digital model of the user's mouth includes:

[0058] Digitally processing the user's oral cavity to obtain an oral cavity model, and segmenting the tooth region of the oral cavity model to obtain the first digital model;

[0059] The user's oral cavity in the occlusal state is digitally processed to obtain an occlusal model, and the tooth area of ​​the occlusal model is segmented to obtain the second digital model.

[0060] In some embodiments, performing model registration on the first digital model and the second digital model to obtain a spatial transformation relationship for mapping the first digital model to the second digital model includes:

[0061] Direction matching is performed on the main axis directions of the first digital model and the second digital model, and model registration is performed based on the first digital model and the second digital model after direction matching to obtain the spatial conversion relationship.

[0062] In some embodiments, the spatial transformation relationship includes a first transformation matrix; performing direction matching on the principal axis directions of the first digital model and the second digital model, and performing model registration based on the first digital model and the second digital model after direction matching to obtain the spatial transformation relationship includes:

[0063] Acquire first point cloud data corresponding to the first digital model, and second point cloud data corresponding to the second digital model;

[0064] spatially flipping the principal axis of the first point cloud data to obtain candidate point cloud data in multiple principal axis directions;

[0065] Performing point cloud registration based on principal component analysis on each candidate point cloud data and the second point cloud data, respectively, to obtain a similarity and a transformation matrix corresponding to each candidate point cloud data, wherein the similarity represents the similarity between the candidate point cloud data and the second point cloud data;

[0066] The candidate point cloud data with the greatest similarity is determined as the target candidate point cloud data, and the transformation matrix corresponding to the target candidate point cloud data is determined as the first transformation matrix.

[0067] In some embodiments, spatially flipping the principal axis of the first point cloud data to obtain candidate point cloud data in multiple principal axis directions includes:

[0068] Processing the first point cloud data to obtain a first eigenvector of the centroid of the first point cloud data, where a direction of the first eigenvector represents a principal axis direction of the first point cloud data;

[0069] The first point cloud data is flipped based on the main axis direction represented by the first eigenvector to obtain candidate point cloud data in multiple main axis directions.

[0070] In some embodiments, performing principal component analysis-based point cloud registration on each candidate point cloud data and the second point cloud data to obtain a similarity corresponding to each candidate point cloud data includes:

[0071] Performing point cloud registration on each candidate point cloud data and the second point cloud data respectively, and calculating an average distance between each candidate point cloud data and the second point cloud data, wherein the similarity includes the average distance;

[0072] The step of determining the candidate point cloud data with the greatest similarity as the target candidate point cloud data includes:

[0073] The candidate point cloud data with the smallest average distance is determined as the target candidate point cloud data.

[0074] In some embodiments, adjusting the occlusal state of the first digital model based on the spatial transformation relationship to obtain a target digital model of the user's oral cavity includes:

[0075] The first point cloud data is spatially transformed according to the first transformation matrix to obtain the target digital model.

[0076] In some embodiments, the spatial transformation relationship further includes a second transformation matrix, and the performing model registration on the first digital model and the second digital model to obtain the spatial transformation relationship for mapping the first digital model to the second digital model further includes:

[0077] Performing sparse iterative closest point point registration on the target candidate point cloud data and the second point cloud data to obtain the second transformation matrix;

[0078] The adjusting the occlusal state of the first digital model based on the spatial transformation relationship to obtain a target digital model of the user's oral cavity includes:

[0079] Performing spatial position transformation on the first point cloud data according to the first transformation matrix to obtain third point cloud data;

[0080] The third point cloud data is spatially transformed according to the second transformation matrix to obtain the target digital model.

[0081] In some embodiments, the first digital model and the second digital model include any one of an upper jaw model, a lower jaw model, an upper and lower jaw model, and a tooth model of the user's mouth.

[0082] In an eighth aspect, embodiments of the present specification provide an occlusal adjustment device, comprising:

[0083] a model acquisition module configured to acquire a first digital model and a second digital model of a user's oral cavity, wherein the first digital model represents a digital model obtained by digitizing the user's oral cavity, and the second digital model represents a digital model obtained by collecting oral data of the user's oral cavity in an occluded state;

[0084] a model registration module configured to perform model registration on the first digital model and the second digital model to obtain a spatial transformation relationship for mapping the first digital model to the second digital model;

[0085] The occlusion adjustment module is configured to adjust the occlusion relationship of the first digital model based on the spatial transformation relationship to obtain a target digital model of the user's oral cavity.

[0086] In some embodiments, the model acquisition module is configured to:

[0087] Digitally processing the user's oral cavity to obtain an oral cavity model, and segmenting the tooth region of the oral cavity model to obtain the first digital model;

[0088] The user's oral cavity in the occlusal state is digitally processed to obtain an occlusal model, and the tooth area of ​​the occlusal model is segmented to obtain the second digital model.

[0089] In some embodiments, the model registration module is configured to:

[0090] Direction matching is performed on the main axis directions of the first digital model and the second digital model, and model registration is performed based on the first digital model and the second digital model after direction matching to obtain the spatial conversion relationship.

[0091] In some embodiments, the model registration module is configured to:

[0092] Acquire first point cloud data corresponding to the first digital model, and second point cloud data corresponding to the second digital model;

[0093] spatially flipping the principal axis of the first point cloud data to obtain candidate point cloud data in multiple principal axis directions;

[0094] Performing point cloud registration based on principal component analysis on each candidate point cloud data and the second point cloud data, respectively, to obtain a similarity and a transformation matrix corresponding to each candidate point cloud data, wherein the similarity represents the similarity between the candidate point cloud data and the second point cloud data;

[0095] The candidate point cloud data with the greatest similarity is determined as the target candidate point cloud data, and the transformation matrix corresponding to the target candidate point cloud data is determined as the first transformation matrix.

[0096] In some embodiments, the model registration module is configured to:

[0097] Processing the first point cloud data to obtain a first eigenvector of the centroid of the first point cloud data, where a direction of the first eigenvector represents a principal axis direction of the first point cloud data;

[0098] The first point cloud data is flipped based on the main axis direction represented by the first eigenvector to obtain candidate point cloud data in multiple main axis directions.

[0099] In some embodiments, the model registration module is configured to:

[0100] Performing point cloud registration on each candidate point cloud data and the second point cloud data respectively, and calculating an average distance between each candidate point cloud data and the second point cloud data, wherein the similarity includes the average distance;

[0101] The step of determining the candidate point cloud data with the greatest similarity as the target candidate point cloud data includes:

[0102] The candidate point cloud data with the smallest average distance is determined as the target candidate point cloud data.

[0103] In some embodiments, the bite adjustment module is configured to:

[0104] The first point cloud data is spatially transformed according to the first transformation matrix to obtain the target digital model.

[0105] In some embodiments, the model registration module is configured to:

[0106] The target candidate point cloud data and the second point cloud data are subjected to point cloud registration based on sparse iterative nearest point to obtain the second transformation matrix.

[0107] In some embodiments, the bite adjustment module is configured to:

[0108] Performing spatial position transformation on the first point cloud data according to the first transformation matrix to obtain third point cloud data;

[0109] The third point cloud data is spatially transformed according to the second transformation matrix to obtain the target digital model.

[0110] In some embodiments, the first digital model and the second digital model include any one of an upper jaw model, a lower jaw model, an upper and lower jaw model, and a tooth model of the user's mouth.

[0111] In a ninth aspect, embodiments of this specification provide an electronic device, including:

[0112] processor; and

[0113] A memory stores computer instructions, wherein the computer instructions are used to enable the processor to execute the method described in any of the above embodiments.

[0114] In a tenth aspect, an embodiment of this specification provides a storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the method described in any of the above embodiments.

[0115] In an eleventh aspect, an embodiment of this specification provides a computer program product, which implements the method described in any of the above embodiments when running.

[0116] The occlusion adjustment method of the embodiment of the present specification includes obtaining a first digital model and a second digital model of the user's oral cavity, performing model registration on the first digital model and the second digital model to obtain a spatial transformation relationship, and adjusting the occlusion relationship of the first digital model based on the spatial transformation relationship to obtain a target digital model. In the embodiment of the present specification, the original first digital model is registered based on the second digital model containing the occlusal state, and the occlusion adjustment of the first digital model is performed based on the spatial transformation relationship obtained by the registration, thereby restoring the actual occlusion relationship represented by the second digital model to the first digital model. This can effectively improve the accuracy of the occlusion adjustment result and make it more consistent with the user's actual occlusion relationship. In addition, no human intervention is required, which simplifies the occlusion adjustment algorithm process, avoids occlusion adjustment deviations caused by subjective factors, and improves the efficiency of the algorithm. At the same time, it is more flexible and can meet the needs of diverse oral restoration scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0117] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0118] FIG1 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

[0119] FIG2 is a schematic structural diagram of a tooth and jaw identification device according to an embodiment of the present invention.

[0120] FIG3 is a schematic structural diagram of a tooth-jaw relationship recognition model in one embodiment of the present invention.

[0121] FIG4 is a schematic structural diagram of a dental model of a tooth to be tested in one embodiment of the present invention.

[0122] FIG5 is a schematic diagram of a partial structure of a tooth-jaw relationship recognition model in another embodiment of the present invention.

[0123] FIG6 is a schematic diagram of the steps of a jaw identification method according to an embodiment of the present invention.

[0124] FIG. 7 is a schematic diagram showing some steps of a first embodiment of a jaw identification method according to an embodiment of the present invention.

[0125] FIG8 is a schematic diagram showing some steps of a second embodiment of a jaw identification method according to an embodiment of the present invention.

[0126] FIG9 is a schematic diagram showing some steps of a third embodiment of a jaw identification method according to an embodiment of the present invention.

[0127] FIG. 10 is a flow chart of an occlusion adjustment method in some embodiments of the present specification.

[0128] FIG. 11 is a flow chart of an occlusion adjustment method in some embodiments of the present specification.

[0129] FIG. 12 is a schematic diagram illustrating a method for adjusting occlusion in accordance with some embodiments of the present disclosure.

[0130] FIG. 13 is a flow chart of an occlusion adjustment method in some embodiments of the present specification.

[0131] FIG. 14 is a schematic diagram illustrating a method for adjusting occlusion in accordance with some embodiments of the present disclosure.

[0132] FIG. 15 is a flow chart of an occlusion adjustment method in some embodiments of the present specification.

[0133] FIG. 16 is a flow chart of an occlusion adjustment method in some embodiments of the present specification.

[0134] FIG. 17 is a flow chart of an occlusion adjustment method in some embodiments of the present specification.

[0135] FIG. 18 is a schematic diagram illustrating a method for adjusting occlusion in accordance with some embodiments of the present disclosure.

[0136] FIG. 19 is a schematic diagram illustrating a method for adjusting occlusion in accordance with some embodiments of the present disclosure.

[0137] FIG20 is a structural block diagram of an occlusal adjustment device in some embodiments of the present specification.

[0138] FIG21 is a structural block diagram of an electronic device in some embodiments of this specification. DETAILED DESCRIPTION

[0139] The present invention will be described in detail below with reference to the specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present invention, and any structural, methodological, or functional changes made by those skilled in the art based on these embodiments are all within the scope of protection of the present invention.

[0140] It should be noted that the term "comprises" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article or apparatus. In addition, the terms "first," "second," "third," "fourth," etc. are used for descriptive purposes only and are not to be understood as indicating or implying relative importance.

[0141] The main idea of ​​this invention is to use relational identification information that characterizes the relative relationship of a tooth pair to determine the jaw information of the two jaws pointed to by the tooth pair. By leveraging the unique local tissue structure of the oral cavity, the problem of jaw identification is transformed into the problem of applying the tooth pair relationship. By leveraging the simplicity and ease of analysis of tooth pair relationships, jaw information can be identified.

[0142] The tooth pair is a combination of at least two corresponding teeth in at least two sets of dental models. The relative relationship between teeth in different dental models can be established based on the corresponding relationship between numbers in tooth position representation methods such as FDI, or can be established based on demand through algorithms or manual recognition.

[0143] Various embodiments, technical principles and corresponding technical effects of the present invention will be further described below with reference to the accompanying drawings.

[0144] An embodiment of the present invention provides an electronic device, as shown in Figure 1. The electronic device may be a computer, a mobile phone, a tablet computer, etc. The present invention is not limited to a specific type of electronic device.

[0145] The electronic device includes at least one processor, at least one memory and a communication bus. The at least one processor and the at least one memory communicate with each other via the communication bus.

[0146] The memory is used to store application programs.

[0147] The processor is configured to implement the steps of a jaw recognition method when executing the application stored in the memory. In one embodiment, the jaw recognition method may include at least one of the following steps:

[0148] respectively obtaining first and second jaw model data of the tooth to be tested, wherein the first and second jaw models correspond to different jaws;

[0149] Obtaining, based on the first jaw model data and the second jaw model data, relationship identification information of a tooth pair to be measured, wherein the tooth pair includes at least one pair of teeth located in different jaws and corresponding to each other, and the relationship identification information is used to indicate the relative positional relationship of the tooth pair;

[0150] According to the relationship identification information, the dental information corresponding to the first dental model data and the second dental model data is determined.

[0151] The tooth and jaw identification method can also be configured based on any technical solution provided below.

[0152] The communication bus may include any number of buses and bridge circuits. In some implementations, in addition to being used to connect the processor and the memory, the communication bus may also be used to connect peripheral devices or other peripheral circuits.

[0153] An embodiment of the present invention provides a jaw identification device, as shown in FIG2 . The jaw identification device includes at least one of the following structures:

[0154] The first module is used to obtain first and second jaw model data of the tooth to be tested; the first and second jaw models correspond to different jaws;

[0155] a second module, configured to obtain, based on the first jaw model data and the second jaw model data, relationship identification information of a tooth pair to be measured, wherein the tooth pair includes at least one pair of teeth located in different jaws and corresponding to each other, and the relationship identification information is used to indicate the relative positional relationship of the tooth pair;

[0156] The third module is used to determine the dental information corresponding to the first dental model data and the second dental model data according to the relationship identification information.

[0157] The use of the jaw recognition device can also be configured based on any of the jaw recognition methods in the technical solutions provided below. Specifically, based on the association relationship between the steps, the related steps can be implemented in the same or different modules.

[0158] The aforementioned apparatus or its modules and units may be implemented as computer chips or physical devices, or as products with corresponding functions. While the aforementioned apparatus is described as being divided into multiple modules, in other embodiments, the functions of the modules may be implemented in the same or multiple software or hardware components.

[0159] One embodiment of the present invention provides a storage medium, which can be specifically a computer-readable storage medium. The storage medium can be set in a computer and store an application program. In this case, the storage medium can be any available medium that can be used by the computer to access data, or can be a storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium such as a floppy disk, hard disk, or magnetic tape, or an optical medium such as a DVD (Digital Video Disc), or a semiconductor medium such as an SSD (Solid State Disk).

[0160] When the application is executed, the steps of a jaw identification method are implemented. In one embodiment, the jaw identification method may include at least one of the following steps:

[0161] respectively obtaining first and second jaw model data of the tooth to be tested, wherein the first and second jaw models correspond to different jaws;

[0162] Obtaining, based on the first jaw model data and the second jaw model data, relationship identification information of a tooth pair to be measured, wherein the tooth pair includes at least one pair of teeth located in different jaws and corresponding to each other, and the relationship identification information is used to indicate the relative positional relationship of the tooth pair;

[0163] According to the relationship identification information, the dental information corresponding to the first dental model data and the second dental model data is determined.

[0164] The storage content of the computer storage medium may also be configured based on the tooth and jaw recognition method in any one of the technical solutions provided below.

[0165] One embodiment of the present invention provides an oral appliance.

[0166] The installation environment of the oral appliance can be the actual inside of a human oral cavity, or a simulated oral entity model or an oral three-dimensional model. The oral appliance can be an actual oral appliance, or a three-dimensional model or entity model of an oral appliance.

[0167] In terms of function and use, the oral device can be a dental appliance or retainer, or it can be an oral device used to train other oral tissues such as orofacial muscle function.

[0168] The oral appliance is manufactured based on jaw information determined by a jaw recognition method. The jaw information can be used to indicate the positional relationship of jaw models, for example, to determine which model corresponds to the upper jaw and / or which model corresponds to the lower jaw among several jaw models. When manufacturing the oral appliance, an adjustment plan can be determined based on the positional relationship of the jaw models. The jaw information can also be used to indicate characteristics of teeth, gums, and other oral tissues.

[0169] In one embodiment, the tooth and jaw identification method includes at least one of the following steps:

[0170] respectively obtaining first and second jaw model data of the tooth to be tested, wherein the first and second jaw models correspond to different jaws;

[0171] Obtaining, based on the first jaw model data and the second jaw model data, relationship identification information of a tooth pair to be measured, wherein the tooth pair includes at least one pair of teeth located in different jaws and corresponding to each other, and the relationship identification information is used to indicate the relative positional relationship of the tooth pair;

[0172] According to the relationship identification information, the dental information corresponding to the first dental model data and the second dental model data is determined.

[0173] The tooth and jaw identification method can also be configured based on any technical solution provided below.

[0174] One embodiment of the present invention provides a display method for outputting a display on a display portion of at least one device body, wherein the device body can be an electronic device such as a computer or a mobile phone, or can also be a virtual reality (VR) device, augmented reality (AR) device, mixed reality (MR) device, etc.

[0175] In one embodiment, the display method includes the steps of: displaying a first jaw model and a second jaw model based on jaw information, wherein the jaw information is obtained based on a jaw recognition method; and the first jaw model and the second jaw model are used to design a correction plan.

[0176] This dental information can be used to indicate the positional relationships of dental models, for example, to determine which model corresponds to the upper jaw and / or the lower jaw among several dental models. When preparing an oral appliance, adjustments can be made based on the positional relationships of the dental models. This information can also be used to indicate characteristics of teeth, gums, and other oral tissues.

[0177] When the jaw information can indicate the positional relationship of the jaw models, the first and second jaw models can be output and displayed at corresponding positions based on the positional relationship. For example, when the first jaw model is an upper jaw model, the first jaw model can be output and displayed at the relatively upper part of the display interface; when the second jaw model is a lower jaw model, the second jaw model can be output and displayed at the relatively lower part of the display interface.

[0178] In one embodiment, the operator can design a treatment plan based on the first and second jaw models by manually adjusting the treatment plan one by one or performing a one-time manual adjustment. In another embodiment, other treatment plan design methods can be automatically implemented to achieve treatment plan design based on the first and second jaw models.

[0179] In one embodiment, the tooth and jaw identification method may include at least one of the following steps:

[0180] respectively obtaining first and second jaw model data of the tooth to be tested, wherein the first and second jaw models correspond to different jaws;

[0181] Obtaining, based on the first jaw model data and the second jaw model data, relationship identification information of a tooth pair to be measured, wherein the tooth pair includes at least one pair of teeth located in different jaws and corresponding to each other, and the relationship identification information is used to indicate the relative positional relationship of the tooth pair;

[0182] According to the relationship identification information, the dental information corresponding to the first dental model data and the second dental model data is determined.

[0183] The tooth and jaw identification method can also be configured based on any technical solution provided below.

[0184] The present invention provides a jaw relationship recognition model. The jaw relationship recognition model is a neural network model trained based on preset tooth pair data and corresponding relationship labels, wherein the relationship labels represent the relative positional relationships of the tooth pairs in the preset tooth pair data.

[0185] The preset tooth pair data can be the data of standard tooth pairs determined in a standard dental model based on rules such as established tooth position representation; it can be the data of actual tooth pairs determined after adaptive adjustment in the actual dental model of the patient or other user; it can also be the data of specific tooth pairs determined by the operator in any dental model according to needs.

[0186] In one embodiment, the tooth-jaw relationship recognition model is used to determine the relative positional relationship between the teeth and jaws, and the corresponding recognition information is used as the tooth-jaw information. The above-mentioned tooth-jaw information can also be of other types.

[0187] The dental relationship recognition model can be carried in any of the above-mentioned storage media, memory, device modules, or method carriers. In one embodiment, the dental relationship recognition model can be implemented as the structure shown in Figure 3, but any deletions, additions, or other improvements to the following structure that do not deviate from the inventive concept can be regarded as derived embodiments thereof.

[0188] The tooth-jaw relationship recognition model includes a first feature extraction module 11. The first feature extraction module 11 is used to extract features from the tooth data to be tested to obtain first tooth data.

[0189] The first feature extraction module 11 can be constructed as a FeatureMap to have a stronger spatial point cloud analysis capability. Specifically, the first feature extraction module 11 at least includes a shared multilayer perceptron (Shared_MLP) in the FeatureMap.

[0190] The first tooth data refers to information data of a tooth on at least one tooth position; the tooth may include only a crown, or may include both a crown and gum, or may include both a crown and a root.

[0191] The tooth data to be measured may include a spatial point cloud, a plane discrete point, a grayscale value distribution, or a pseudo-color distribution.

[0192] The first feature extraction module 11 can also be used to obtain first feature data corresponding to the first tooth data. The first feature data can be obtained by processing the first tooth data, and the feature extraction method can be at least one of SIFT (Scale-Invariant Feature Transform), HOG (Histogram of Oriented Gradient), traditional CNN (Convolutional Neural Networks), SURF (Speeded Up Robust Features), ORB (Oriented FAST and Rotated BRIEF), and LBP (Local Binary Patterns).

[0193] The first feature extraction module 11 can be constructed as a FeatureMap. Specifically, the first feature extraction module 11 at least includes a maximum pooling layer (Max Pool) in the FeatureMap.

[0194] The tooth-jaw relationship recognition model includes a second feature extraction module 12. The second feature extraction module 12 is used to extract features from the tooth data to be tested to obtain second tooth data.

[0195] The second dental data and the second feature data can be identical or similar to the first dental data and the first feature data in definition and configuration, respectively; the architecture, algorithm or included operators of the second feature extraction module 12 can also be identical or similar to those of the first feature extraction module 11.

[0196] The tooth pointed to by the first tooth data may have an occlusal relationship with the tooth pointed to by the second tooth data. The first tooth data and the second tooth data have an occlusal relationship. In other words, the second feature extraction module 12 can be used to receive the second tooth data that has an occlusal relationship with the first tooth data; alternatively, the first feature extraction module 11 can be used to receive the first tooth data that has an occlusal relationship with the second tooth data.

[0197] In some embodiments, the two tooth data may refer to two teeth located in the upper and lower jaws, respectively, but not in an occlusal relationship with each other, or may refer to two teeth located in different upper and lower jaws. For the former, the positional relationship between the upper and lower jaws can still be determined based on the relationship identification information corresponding to the tooth data. For the latter, the corresponding jaw information corresponding to the first and second jaw model data can also be determined based on the corresponding relationship identification information.

[0198] When the jaw information is used to indicate the positional relationship, the two jaws may have an occlusal relationship and one of them is the upper jaw and the other is the lower jaw, the two jaws may be the upper jaw and the lower jaw respectively but do not have an occlusal relationship, the two jaws may be two groups of identical or different upper jaws, or the two jaws may be two groups of identical or different lower jaws.

[0199] The tooth-jaw relationship recognition model includes a splicing layer 21. The splicing layer 21 is used to combine the first feature data and the second feature data to generate tooth pair feature data.

[0200] The combination can be a simple concatenation, for example, arranging the first feature data in matrix form on the left and the second feature data in matrix form on the right; or the tooth pair feature data can be constructed by weighted fusion or parameter-based operation. The tooth pair feature data contains information about both the first tooth data and the second tooth data.

[0201] The tooth-jaw relationship recognition model includes a secondary fully connected layer 22. The secondary fully connected layer 22 is used to determine the relative positional relationship between the first tooth and the second tooth based on the tooth pair feature data and generate relationship identification information. In a specific embodiment, the relationship identification information can represent at least one of the following information:

[0202] (1) The first tooth is relatively above, and the second tooth is relatively below;

[0203] (2) The first tooth is relatively lower and the second tooth is relatively higher;

[0204] (3) The first and second teeth are both relatively high;

[0205] (4) The first and second teeth are relatively lower;

[0206] (5) There is an occlusal relationship between the first tooth and the second tooth;

[0207] (6) There is no occlusal relationship between the first tooth and the second tooth.

[0208] It can be understood that when applying the FDI tooth position representation method, an occlusal relationship indicates that the two teeth have the same second "position position" and the first "quadrant position" corresponds to each other (2 corresponds to 3, 1 corresponds to 4). No occlusal relationship can include any of the following situations: being located in the same jaw but without an occlusal relationship (for example, tooth 11 and tooth 12), being located in different jaws (for example, tooth 11 in one jaw and tooth 41 in another jaw).

[0209] An occlusal relationship can be shown in Figure 4. For example, when the first tooth is tooth number 12 t12 located in quadrant I and the second tooth is tooth number 42 t42 located in quadrant IV, the tooth data corresponding to the two teeth form tooth pair data, which in this case refers to tooth pair Pair 12-42. Similarly, quadrant I can also include at least a portion of teeth number 11 t11 through 18 t18 in Figure 4; quadrant II can also include at least a portion of teeth number 21 t21 through 28 t28 in Figure 4; quadrant III, corresponding to quadrant II, can also include at least a portion of teeth number 31 t31 through 38 t38 in Figure 4; and quadrant IV, corresponding to quadrant I, can also include at least a portion of teeth number 41 t41 through 48 t48 in Figure 4.

[0210] The relationship identification information is used to determine the relative positional relationship between the first jaw where the first tooth is located and the second jaw where the second tooth is located. When using the relationship identification information, or executing a jaw identification method using the relationship identification information, at least one of the following information can be generated:

[0211] (1) The first jaw is the upper jaw, and the second jaw is the lower jaw;

[0212] (2) The first jaw is the mandible, and the second jaw is the maxilla;

[0213] (3) Both the first and second jaws are maxillary;

[0214] (4) Both the first and second jaws are mandibular;

[0215] (5) The first and second jaws can be combined to form a complete jaw face;

[0216] (6) The first and second jaws cannot be combined to form a complete jaw face.

[0217] The jaw relationship recognition model includes at least one of a first feature extraction module 11, a second feature extraction module 12, a concatenation layer 21, and a secondary fully connected layer 22. In a preferred embodiment, the jaw relationship recognition model includes all of the aforementioned structures. In an even more preferred embodiment, the jaw relationship recognition model is constructed based on the aforementioned structures to form a PointNet architecture to better achieve the corresponding technical effects.

[0218] In the above preferred embodiment, the first feature extraction module 11 and the second feature extraction module 12 can be set in parallel, and the two respectively send their outputs to the splicing layer 21, and the splicing layer 21 further sends the processed data to the secondary fully connected layer 22. In this way, the final prediction output or final inference output in the form of relationship identification information is obtained from the side of the secondary fully connected layer 22. The second feature extraction module 12 and the first feature extraction module 11 have the same configuration. In the above preferred embodiment, the parallel first feature extraction module 11 and the second feature extraction module 12 constitute FeatureMap as the front-end network structure of the PointNet architecture. Compared with the traditional PointNet architecture, it can adapt to the paired input of tooth data and perform comprehensive processing to generate an output that can reflect the relationship between the two teeth. That is, the two tooth data in the tooth pair are respectively input into the first feature extraction module 11 and the second feature extraction module 12, and then the output reflecting the relationship between the two teeth is obtained through the splicing layer 21 and the fully connected layer 22.

[0219] As shown in Figure 5, the secondary fully connected layer 22 may include at least one group of multi-layer perceptrons (MLP). On the one hand, the secondary fully connected layer 22 may include several multi-layer perceptrons with layer sizes of 512, 256 and 2, respectively. The layers of the three sizes may be set in sequence. By introducing hidden layers, nonlinear problems can be handled, and feature extraction, conversion and information reorganization can be easily performed, which is suitable for a variety of classification problems. On the other hand, the multi-layer perceptrons of the secondary fully connected layer 22 can be used to constitute a Shared_MLP (shared multi-layer perceptron), thereby reducing the training parameters of the network and realizing a weight sharing mechanism similar to a convolutional neural network. The perceptrons configured with sizes of 512, 256 and 2 can perform step-by-step feature extraction on larger input data in sequence, which can increase the accuracy of the operation.

[0220] In one embodiment, the first feature extraction module 11 may include two components for outputting features of different data volumes. For example, the first feature extraction module 11 includes a first fully connected layer 111 and / or a maximum pooling layer 112. In one embodiment, the first feature extraction module 11 includes both the first fully connected layer 111 and the maximum pooling layer 112, and the two are arranged sequentially.

[0221] The first fully connected layer 111 may include at least one set of multilayer perceptrons. In one embodiment, the first fully connected layer 111 includes multiple multilayer perceptrons with layer sizes of 6, 64, 128, and 1024. When the first input node Input point 1 of size n×6 forms the input of the first feature extraction module 11, an input of size 1024 can be gradually constructed based on the input of size 6, thereby improving the dimensionality and accuracy of feature processing.

[0222] The multiple multilayer perceptrons of the first fully connected layer 111 can collectively constitute a Shared_MLP. After the size of the first input node Input point1 is adjusted, the structure of the first fully connected layer 111 can also be adjusted accordingly.

[0223] When the second feature extraction module 12 has the same structure as the first feature extraction module 11, the second feature extraction module 12 can construct an output of size 1024 based on the second input node Input point2 of size n×6 through Shared_MLP(6,64,128,1024) and the maximum pooling layer Max Pool. After the output data corresponding to the two tooth data or tooth feature data are spliced, intermediate data of size 2048 is formed at the splicing layer 21. Then, it is reduced to 512, 256, and 2 respectively through the secondary fully connected layer, ultimately forming a relationship identification information output of size 2.

[0224] In one embodiment, the 6 in the n×6 dimension of the two input nodes can be the 6 coordinate values ​​corresponding to n points in the tooth data to be measured. Three of the coordinate values ​​are the x, y, and z coordinates of the vertex; the other three coordinate values ​​are the coordinate values ​​of the normal vector of the plane (which can be a patch) corresponding to the vertex.

[0225] By configuring the feature extraction, concatenation, and classification layers separately, and employing multi-layer perceptrons for scale expansion and reduction at the feature extraction and classification layers, the model is more adaptable to the input data and ensures operational stability, which is particularly evident in point cloud processing scenarios. Furthermore, the parallelization of the two feature maps allows for the unique case of tooth pair processing and improves overall processing efficiency.

[0226] As shown in Figure 6, one embodiment of the present invention provides a method for tooth and jaw recognition. The corresponding application or instructions of this method can be installed in the above-mentioned electronic device, tooth and jaw recognition device and / or storage medium, or can be installed in a carrier of a display method, or in a carrier for implementing oral appliance preparation, to achieve the corresponding technical effects. The tooth and jaw recognition method can specifically include the following steps.

[0227] Step S1 : obtaining first and second jaw model data of a tooth to be tested, respectively. The first and second jaw models correspond to different jaws.

[0228] The first and second jaw model data may refer to the upper and lower jaws of the same subject (patient) in an occlusal relationship, or to different jaws of different subjects, or to two upper jaws or two lower jaws. Preferably, the first and second jaw model data refer to the upper and lower jaws of the same subject (patient) in an occlusal relationship, or to different jaws of different subjects.

[0229] Acquiring can refer to any means of ensuring that the first and second jaw model data are present on a carrier executing the jaw recognition method. For example, the jaw model data can be obtained or received externally, or retrieved from internal storage. Alternatively, the jaw model data can be obtained through some form of data processing. The acquisition described herein is subject to the aforementioned explanation and will not be further elaborated upon.

[0230] Step S2: obtaining the relationship identification information of the tooth pair to be measured based on the first jaw model data and the second jaw model data.

[0231] The tooth pair includes at least one pair of teeth located in different jaws and corresponding to each other.

[0232] For example, the first dental jaw model data includes first tooth data, and the second dental jaw model data includes second tooth data. The first tooth data and the second tooth data correspond to each other, and the first tooth and the second tooth pointed to by the two are used to constitute the tooth pair, and the first tooth data and the second tooth data can be the tooth pair data.

[0233] The first tooth data and the second tooth data can be specifically explained as: the tooth position or the first tooth represented by the first tooth data is located in the first jaw pointed to by the first jaw model data, and the tooth position or the second tooth represented by the second tooth data is located in the second jaw pointed to by the second jaw model data.

[0234] When implementing this specific example, a corresponding relationship can be established between the first tooth data and the second tooth data. This means that an occlusal relationship exists between the first tooth referenced by the first tooth data and the second tooth referenced by the second tooth data. This naturally avoids errors caused by selecting tooth data located in the same upper jaw or the same lower jaw as input. If "both located in the upper jaw" or "both located in the lower jaw" is the desired output of the tooth and jaw recognition method, then there is no need to set this occlusal relationship, and no error is expected.

[0235] The tooth pair may be provided in only one group, corresponding to one piece of tooth pair data. In this case, the first tooth data and the second tooth data corresponding to a single first tooth and a single second tooth constitute a group of tooth pair data.

[0236] The tooth pair can be provided in multiple groups, corresponding to multiple pieces of tooth pair data. In this case, the multiple groups of first teeth and second teeth corresponding to each other correspond to multiple groups of first tooth data and second tooth data, forming multiple groups of tooth pair data.

[0237] The relationship identification information is used to indicate the relative position relationship of the tooth pair.

[0238] The relationship identification information may be a symbol, a label, data, information, etc. representing the relative position relationship between the first tooth and the second tooth.

[0239] The relationship identification information can be derived from manual labeling or automated processing by algorithms or program instructions. The latter can specifically be derived from executing point cloud semantic segmentation or from a classification algorithm of a neural network model.

[0240] Step S3: determining the dental information corresponding to the first dental model data and the second dental model data according to the relationship identification information.

[0241] In this way, by applying the tooth-jaw recognition method provided by the present invention, the tooth-jaw information corresponding to the teeth can be determined based on the positional relationship between two teeth in the tooth pair determined by the relationship identification information.

[0242] The dental information includes maxillary determination information or mandibular determination information of the first dental model, and maxillary determination information or mandibular determination information of the second dental model.

[0243] In this way, the upper and lower jaws can be identified and distinguished quickly and accurately.

[0244] It is understandable that the technical solution provided by the present invention can also achieve the technical effects of separately identifying and determining the upper jaw, separately identifying and determining the lower jaw, and separately identifying and determining whether the upper and lower jaws belong to the same jaw face. In the case where the first tooth data and the second tooth data refer to data information of the same tooth in different orthodontic states, and the relationship identification information at least records the order of the first tooth data and the second tooth data, the technical solution provided by the present invention can also achieve the effect of determining the order of the dental model data and assisting in the construction of the orthodontic appliance.

[0245] In the first embodiment provided by the present invention, as shown in FIG7 , the specific implementation process of step S2 in the tooth and jaw recognition method is further described, including the following steps.

[0246] Step S21 : obtaining tooth pair data of at least one pair of teeth located in different jaws and corresponding to each other based on the first jaw model data and the second jaw model data.

[0247] The method of determining the tooth pair data can be achieved through a preset feature recognition algorithm, especially a point cloud feature extraction algorithm; it can also be obtained by constructing a trained neural network model for prediction. The prediction target can mainly be a crown point cloud data set, which is used as the tooth pair data.

[0248] Step S22: Input the tooth pair data into the tooth-jaw relationship recognition model in parallel to predict and obtain corresponding relationship identification information.

[0249] The tooth-jaw relationship recognition model is a neural network model, and the tooth-jaw relationship recognition model is trained based on preset tooth pair data and relationship labels that represent the relative positional relationship of the tooth pairs in the preset tooth pair data.

[0250] In one case, one of the two sets of to-be-measured dental and jaw data points to the upper jaw, and the other one points to the lower jaw.

[0251] For example, the first jaw model data points to the upper jaw and the second jaw model data points to the lower jaw. Thus, the first tooth pointed to by the first tooth data is located in the upper jaw, and the second tooth pointed to by the second tooth data is located in the lower jaw. The relationship identification information corresponds to indicating that the tooth pointed to by the first tooth data is located relatively above the tooth pointed to by the second tooth data.

[0252] The term "parallel input" can be understood as inputting the tooth pair data separately into the jaw relationship recognition model, thereby improving the parallel processing capability for multiple tooth pair data. The term "parallel input" can also be understood as using different tooth data within the tooth pair data as multiple parallel inputs to a single jaw relationship recognition model, thereby improving the processing capability for a single set of tooth pair data. This facilitates the use of such input to implement configurations such as mutual supervision and weight sharing, improves overall processing efficiency, and facilitates the prediction and generation of relationship identification information.

[0253] In one embodiment, the tooth-jaw relationship recognition model includes at least one of the following structures.

[0254] The first feature extraction module 11 and the second feature extraction module 12 are configured to receive tooth data input in parallel;

[0255] A splicing layer 21 is used to combine the tooth pair feature sequences output by the first feature extraction module 11 and the second feature extraction module 12;

[0256] The secondary fully connected layer 22 is used to predict the relationship identification information based on the tooth pair feature sequence.

[0257] The dental relationship recognition model can include all of the above-mentioned structures. The specific scheme of the above-mentioned structure can be configured with reference to the technical scheme provided above, especially it can be configured as shown in Figure 5, which will not be repeated here. It is worth noting that when the first feature extraction module 11 or the second feature extraction module 12 includes a primary fully connected layer 111, the "primary" and the "secondary" only have the connotation of sequence, and do not necessarily refer to the number of layers set by the fully connected layer; when the primary fully connected layer 111 is not included, the secondary fully connected layer 22 can be expressed as a "fully connected layer".

[0258] The following further introduces the data processing flow of the tooth-jaw relationship recognition model. In one embodiment, the tooth-jaw recognition method includes the following steps.

[0259] Step S221: performing feature extraction on the tooth data to be tested according to the first feature extraction module to obtain first tooth data; performing feature extraction on the tooth data to be tested according to the second feature extraction module to obtain second tooth data.

[0260] Step S222: obtaining first feature data and second feature data corresponding to the first tooth data and the second tooth data respectively, and generating a tooth pair feature sequence.

[0261] Step S223: predicting the relationship identification information between the first tooth and the second tooth based on the tooth pair feature sequence.

[0262] In this way, by constructing a parallel input dental relationship recognition model, it is not only possible to receive a larger amount of data for processing at a single time, but also possible to perform feature extraction in parallel to facilitate the fusion and mutual comparison of relationship identification information.

[0263] The jaw relationship recognition model can be constructed based on a variety of architectures. In one embodiment, the PointNet architecture is used to better adapt to feature extraction scenarios with global features of point clouds. The PointNet architecture mentioned in this article not only includes traditional PointNet, but also PointNet++ or F-PointNet. Based on this, the jaw relationship recognition method includes the steps of: constructing a neural network model based on the PointNet architecture.

[0264] The jaw relationship recognition model is based on a neural network model architecture and is trained using pre-set tooth pair data and their pre-set relationship labels as training and test sets, or as a training, validation, and test set. In one embodiment, the jaw relationship recognition method further includes configuring the optimizer to Adam and the learning rate to 0.001. This stabilizes the training speed during the iteration process, maintaining the stability and accuracy of the model parameters.

[0265] In one embodiment, the jaw recognition method may specifically include configuring the loss function as a cross entropy loss function.

[0266] Before calculating the cross entropy loss function, label smoothing can be used to improve the quality of the calculation results.

[0267] The number of single samples in training, BatchSize, can be configured as 32.

[0268] In one embodiment, the jaw recognition method further includes the steps of: configuring the optimizer to be Adam, configuring the learning rate to be 0.001, and configuring the loss function to be a cross entropy loss function according to the preset relationship label and the preset tooth pair data, and performing training on the neural network model.

[0269] The preset relationship tag corresponds to the relationship identification information, characterizing the relative position relationship between a number of teeth, and may indicate "upper and lower", "lower and upper", "same as above", "same as below", "with occlusion", "no occlusion" and the like.

[0270] In the third embodiment provided by the present invention, as shown in FIG9 , the jaw identification method further includes the following steps.

[0271] Step S211 , dividing the first jaw model and the second jaw model to obtain a plurality of first jaw tooth data and a plurality of second jaw tooth data.

[0272] The segmentation can be based on the first jaw model data and the second jaw model data, and several teeth are segmented to obtain first jaw tooth data containing tooth data corresponding to at least one tooth (first tooth data), and second jaw tooth data containing tooth data corresponding to at least one tooth (second tooth data).

[0273] The segmentation is not limited to the process of segmenting the dental model to obtain teeth, but may also include the step of processing the dental model data. The data processing of the dental model is used to improve the accuracy and efficiency of the segmentation process.

[0274] Step S212 , matching the first jaw tooth data and the second jaw tooth data according to the tooth position numbers to obtain tooth pair data.

[0275] In one embodiment, the tooth and jaw recognition method may further include a data sampling process.

[0276] Data sampling is used to mark and process teeth, and based on the mark and quadrant relationship, tooth pairs are constructed and finally tooth pair data is formed. The present invention does not exclude the combination of manual marking and tooth pairing.

[0277] The above step S211 can be understood as the model segmentation process in the tooth and jaw recognition method, and step S212 can be understood as the data sampling process in the tooth and jaw recognition method.

[0278] In one embodiment of the present invention, the jaw recognition method further includes the following steps before performing model segmentation.

[0279] Step S2111: simplifying the surfaces of the first and second jaw models according to the degree of differentiation of the parts in the jaw models.

[0280] The degree of distinction of the site includes the degree of distinction between the site and other intraoral tissues, and / or the degree of distinction between adjacent sites.

[0281] The surface simplification can be quadratic surface simplification. This surface simplification is used to reduce the overall data volume of the dental model data to improve efficiency. During the surface simplification process, different weights can be assigned to different areas of the dental model data to prevent the loss of important features such as the gum line and crown information, and to prevent key locations from being simplified.

[0282] Step S2112: respectively register the first dental model and the second dental model, and segment the first dental model and the second dental model based on the tooth segmentation model.

[0283] The registration of the dental model may include coarse registration and / or fine registration.

[0284] In one embodiment, the jaw recognition method includes sequentially performing coarse registration and fine registration on the first jaw model. In one embodiment, the jaw recognition method includes sequentially performing coarse registration and fine registration on the second jaw model.

[0285] Model registration is used to improve the overall accuracy of dental model data based on a preset standard model, facilitating subsequent predictions.

[0286] The above step S2111 can be understood as the surface simplification process in the tooth and jaw recognition method, and step S2112 can be understood as the model registration process in the tooth and jaw recognition method.

[0287] In an embodiment in which surface simplification and model registration are sequentially performed on the dental model data, the input data for performing the model registration may specifically be the output data of the surface simplification.

[0288] In an embodiment in which model registration and model segmentation are sequentially performed on the dental model data, the input data for performing the model segmentation may specifically be the output data of the model registration process.

[0289] In an embodiment in which model segmentation and data sampling are sequentially performed on the dental model data, the input data for performing the data sampling may specifically be the output data of the model segmentation process.

[0290] The above embodiments can be separated or combined. The tooth and jaw recognition method can include the steps of performing at least one of surface simplification, model registration, model segmentation, and data sampling on two sets of tooth and jaw model data.

[0291] Specifically, the above four implementations can be combined arbitrarily, and can be combined in pairs or in groups of three. Moreover, under the premise of ensuring normal input and output, the order can also be adjusted as needed, which will not be repeated here.

[0292] In one embodiment, the dental model recognition method may include the steps of sequentially performing surface simplification, model registration, model segmentation, and data sampling on one of the two sets of dental model data. This sequentially removes edge regions or other unimportant areas that have little impact on the generation of relationship identification information; aligns and improves the overall accuracy of the model; segments the model to form tooth pairs and samples data that characterizes the morphology and other features of the tooth pairs; and ultimately generates tooth pair data sufficient for accurate computation.

[0293] In one embodiment, the jaw recognition method may include the steps of sequentially performing surface simplification, model registration, model segmentation, and data sampling on two sets of jaw model data. In this manner, the two sets of jaw model data are subjected to the same pre-processing, resulting in good consistency between the generated tooth data used to form the tooth pair data.

[0294] In an embodiment provided by the present invention, the surface simplification step may specifically include the following steps.

[0295] Step P1: Determine the face segmentation method of the dental model according to the degree of distinction between adjacent facets in the dental model, and obtain the facets of the dental model accordingly.

[0296] Step P2: merge the valid vertices of the obtained facets.

[0297] In one embodiment, the valid vertices of the obtained facets may be merged based on the merging error, so that subsequent tooth pair data processing can be more accurate.

[0298] The determined face segmentation method may specifically be a simplified face segmentation method, so as to reduce the number of face segments obtained by segmentation.

[0299] In one embodiment provided by the present invention, the surface simplification step included in the jaw recognition method may specifically include the steps of: determining the face segmentation method of the jaw model based on the degree of distinction between the face and other oral tissues, and the cost of adjacent facets belonging to the same oral tissue.

[0300] The degree of distinction between the surface film and other intraoral tissues is used to distinguish between tooth surface films and other intraoral tissue surface films.

[0301] The cost of assigning adjacent surface patches to the same intraoral tissue is used to measure whether the current operation of assigning adjacent surface patches to the same intraoral tissue will result in surface patches that originally point to different intraoral tissues being mistakenly assigned to the same intraoral tissue.

[0302] The degree of distinction between the face sheet and other intraoral tissues is determined based on at least one of the following indicators:

[0303] (1) The degree to which the current facet is close to the cutting edge line;

[0304] (2) the degree to which the current facet is close to the incisal extreme point of the corresponding tooth;

[0305] (3) The degree to which the current facet is away from the model center of the dental model data.

[0306] An embodiment of the present invention provides a solution corresponding to the above surface simplification process, which specifically includes the following steps.

[0307] Step P11, calculating the Q matrix of the vertices in the dental model data.

[0308] The Q matrix can be interpreted as a Quadric matrix, that is, a quadratic surface matrix.

[0309] The technical solution for calculating the Q matrix can be selected according to the needs of those skilled in the art. Here, a more efficient technical solution is provided for reference. In this technical solution, step P11 may include the following steps.

[0310] Step P111, determine the vertex and the reference plane where the vertex is located.

[0311] Step P112: Determine the plane equation of the reference plane and calculate the quadratic error surface matrix of the vertices.

[0312] Step P113, iteratively obtain the quadratic error surface matrix belonging to the current plane set in the dental model data, sum the quadratic error surface matrices, and obtain the Q matrix corresponding to the vertex.

[0313] The reference plane is a plane formed by a vertex and a preset number of vertices located around it.

[0314] Specifically, the plane equation of the reference plane may have the following form:

[0315] ax+by+cz+d=0; where a 2 +b 2 +c 2 =1.

[0316] Then, the plane equation has the form p = [abcd] T Based on this, the quadratic error surface matrix corresponding to the vertex and the reference plane can have the following form:

[0317] Based on this, the quadratic error surface matrix K p The Q matrix can have the following form: Q = ∑p∈planes(v)K p .

[0318] planes(v) represents the current set of planes corresponding to this vertex.

[0319] Regarding the valid vertices of the facets obtained by merging in step P2, a specific implementation method may include the following steps.

[0320] Step P21, determining the merging error of the valid vertex combination.

[0321] A valid vertex can be interpreted as one that satisfies the following two conditions:

[0322] (1) There is an edge between two vertices;

[0323] (2) The distance between two vertices is less than a certain threshold; specifically, define v1 and v2 as two vertices, then the second condition is |v1-v1|<t, where t is the set threshold.

[0324] Step P22: Select the valid vertex combination with the minimum merging error and update the merging error of the relevant valid vertices.

[0325] In order to avoid connecting vertices that are far away from each other, the threshold needs to be set dynamically following the number of vertex combinations. In this process, it is necessary to consider the algorithm of the iterative collapse itself and the merging error Δv between the target vertex and the original vertex. In this way, the optimal iterative collapse point can be found by minimizing the merging error Δv.

[0326] The combined error Δv must at least satisfy the following formula: Δv=v T Q(v)v.

[0327] The iterative collapse process can be expressed as (v1, v2) → v, where the first vertex v1 and the second vertex v2 are the original vertices, and v is the target vertex. Q(v) is the Q matrix of the target vertex v. In the process of finding the target vertex v, the arithmetic mean of the first vertex v1 and the second vertex v2 can be used as the initial target vertex, and then the optimal target vertex can be found through iteration.

[0328] In one embodiment, the optimal target vertex can be quickly determined by Q matrix operation. The process of finding the minimum value is abstracted into a linear problem. That is:

[0329] Assuming that this matrix is ​​reversible, the optimal target vertex can be directly calculated according to the following formula:

[0330] If this matrix is ​​not invertible, the optimal target vertex can be found at a line segment, an endpoint, or a midpoint formed by connecting the first vertex v1 and the second vertex v2.

[0331] The process of determining the valid vertex combination with the minimum merge error can also be abstracted as a stack problem. Specifically, the valid vertex combinations (or valid vertex pairs) can be placed in a stack with the error metric being the merge error Δv as the key value, so that the minimum cost pair is at the top of the stack.

[0332] Remove the valid vertex pair with the smallest merge error Δv from the heap, such as (v1, v2), collapse the valid vertex pair, and update the merge error Δv of the valid vertex pair related to the first vertex v1 (that is, the current vertex). In this way, the surface is simplified and vertices and patches that do not need to be distinguished from each other are merged from the vertex perspective.

[0333] To further reduce the number of facets and computational complexity, during surface simplification, the facets of the dental model data can be simplified before calculating the Q matrix. This can focus on reducing the data volume in certain areas and improve computational efficiency without sacrificing important features. Specifically, step P11 may include determining a facet segmentation method for the dental model and calculating the Q matrix for the vertices in the simplified dental model data.

[0334] In one embodiment, the patch segmentation strategy may be constrained by the overall energy function, thereby simplifying the patch segmentation strategy, reducing the number of patches, or even minimizing the number of patches.

[0335] Based on this, the tooth and jaw identification method may specifically include the following steps.

[0336] Step P1101: Fitting the overall energy function of the facets in the dental model data.

[0337] Step P1102, minimizing the overall energy function to obtain simplified jaw data to be measured.

[0338] The overall energy function is used to evaluate whether the current degree of simplification is optimal; it includes a patch probability function corresponding to each patch and a neighbor discrimination function corresponding to two adjacent patches. Furthermore, the patch probability function is used to distinguish intraoral tissue regions, and the neighbor discrimination function represents the cost of assigning adjacent patches to different intraoral tissue regions.

[0339] In this way, the degree of patch simplification can be evaluated from two dimensions: the inherent properties of the oral tissue area, especially the importance of the area, and the ability of adjacent patches to be merged within the allowable range. This provides the possibility of customization while maintaining the optimal simplification strength.

[0340] The overall energy function may be determined by weighted calculation.

[0341] In one embodiment, the overall energy function is equal to the weighted sum of the face probability function and the adjacent discrimination function. For example, the overall energy function is defined as E(x), the set of facets in the dental model data is δ, and the facet probability function of the i-th facet is E unary (x i ), the set of adjacent facets in the dental model data is The adjacent distinction function between the i-th patch and the j-th patch is E pairwise (x i ,x j ), then the overall energy function E(x) can be configured to at least satisfy the following formula:

[0342] λ is the adjacent discrimination function E pairwise (x i ,x j ) relative to the overall energy function E(x).

[0343] In one embodiment, the patch probability function can be set based on the spatial position of the patch in the dental model data. This means that step P1101 specifically includes: fitting the patch probability function of the patch in the dental model data based on the spatial position of the patch; and determining the overall energy function based on the patch probability function.

[0344] Specifically, a higher probability function can be set for the facets close to the crown, close to the overall center of the model, or close to teeth No. 11, No. 21, No. 31, and No. 41, which is equivalent to giving them higher weights or lower simplification requirements to prevent feature loss due to over-simplification.

[0345] The patch probability function may be determined in a weighted manner based on several probability energies. In one embodiment, the patch probability function may be the sum or weighted sum of several probability energies.

[0346] In this way, by setting the probability energy, the patch probability function can be given the ability to consider various spatial positions of the patch. In a preferred embodiment, the patch probability function can include at least one of the first probability energy, the second probability energy and the third probability energy.

[0347] The first probability energy represents the degree to which the current facet is close to the incisal line. The incisal line can be interpreted as a line formed by fitting the extension line of the incisal end of the crown, or as a line formed by fitting the projection line of the crown section onto the labial or lingual plane. Specifically, the incisal line can be interpreted as a smile line in the anterior region and as a Spee curve in the posterior region.

[0348] The second probability energy characterizes the degree to which the current facet is close to the incisal extreme point of the corresponding tooth. The incisal extreme point can be interpreted as the point on the incisal tip or cutting surface that is farthest from the root of the tooth position. Taking the maxillary as an example, if the root is relatively above the crown, the incisal extreme point can be interpreted as the lowest point on the z-axis of the incisal tip or cutting surface of a certain tooth position; if the root is relatively below the crown, the incisal extreme point can be interpreted as the highest point on the z-axis of the incisal tip or cutting surface of a certain tooth position.

[0349] The third probability energy represents the degree to which the current facet is away from the model center of the dental model. The definition of the model center depends on the importance of the posterior tooth area to subsequent tasks.

[0350] If the posterior and anterior areas are of similar importance, the model center can be set between the upper and lower jaws. If the posterior area is significantly less important than the anterior area, the model center can be set at the intersection of the mesial extensions of teeth 11, 21, 31, and 41. When the model center is set between the upper and lower jaws, the probability energy-based adjustment of the mask weights can remove interference from the lingual gingiva.

[0351] If the patch probability function includes the first probability energy, the second probability energy and the third probability energy at the same time, then the patch probability function can be set to be equal to the weighted sum of the first probability energy, the second probability energy and the third probability energy.

[0352] Assume that the z-axis is the vertical coordinate axis in the dental model, and the origin of the coordinate is the center of the model. Then:

[0353] (1) The first probability energy can be configured to set weights for the facets from large to small as the probability energy in the direction of increasing absolute value of the z-axis. That is, the probability energy of the facets close to the cutting end line is greater than the probability energy of the facets far from the cutting end line.

[0354] (2) The second probability energy can be configured by starting from the incisal extreme point of the tooth position, expanding outward, and assigning weights from large to small to the facets as the probability energy. That is, the probability energy of the facets close to the incisal extreme point is greater than the probability energy of the facets far from the incisal extreme point.

[0355] (3) The third probability energy can be configured to calculate the distance between the patch and the model center and assign weights from small to large to the patches as the probability energy according to the distance from near to far. That is, the probability energy of the patch close to the model center is smaller than the probability energy of the patch far from the model center.

[0356] Based on this, the sum of the absolute values ​​of the weight values ​​of the first probability energy, the second probability energy, and the third probability energy can be configured to be 1. The weight value of the third probability energy has an opposite sign to the weight value of the first probability energy and the weight value of the second probability energy.

[0357] The patch probability function can be defined as E unary (Or define that for a certain patch, the patch probability function value is E unary; The probability energy E1, E2, E3 and so on are similarly defined below), and the first probability energy is defined as E1, the weight value of the first probability energy is ε1, the second probability energy is defined as E2, the weight value of the second probability energy is ε2, the third probability energy is defined as E3, the weight value of the third probability energy is ε3. Then the patch probability function is E unary And the weight value at least meets:

[0358] It can be understood that the weight values ​​ε1, ε2 and ε3 are the three probability energies relative to the patch probability function E unary The probability energies E1, E2 and E3 set the weights between the slices of the dental model data, and the two can be interpreted differently.

[0359] Patch probability function E unary The larger the value, the greater the possibility that the facet belongs to a tooth (especially a crown). Furthermore, at least two classification thresholds can be set according to the facet probability function E. unary , the surface of the jaw model to be tested is divided into three simplified regions. i It can correspond to the i-th patch, based on the patch probability function E unary The meaning of the attributed type.

[0360] In one embodiment, the adjacent discrimination function can be set based on the angular distance between adjacent facets and / or the capacity function value. Equivalently, step P1101 specifically includes: fitting the adjacent discrimination function of adjacent facets in the dental model data based on the angular distance between adjacent facets and / or the capacity function; and determining the overall energy function based on the adjacent discrimination function.

[0361] In one embodiment, a capacity function can be calculated based on the angular distance value, and the adjacent discrimination function can be calculated based on the capacity function. The capacity function can even be directly used as the adjacent discrimination function. Specifically, for example, if the adjacent discrimination function includes a first discrimination function corresponding to the first face patch and the second face patch, the jaw recognition method can specifically include the following steps.

[0362] Step P11011: Calculate a first angular distance between the first facet and the second facet according to the normal angle between the first facet and the second facet.

[0363] Step P11012: Execute the maximum flow minimum cut according to the first angular distance and the average angular distance, calculate the first capacity function of the first facet and the second facet, and use the first capacity function as the first discrimination function.

[0364] The first angular distance is used to evaluate the similarity between the first and second facets. The greater the first angular distance (the larger the value), the less similar the two facets are, and vice versa. Based on the similarity, a capacity function can be calculated based on the average angular distance between adjacent facets to evaluate the cost of fusing the two facets. The greater the similarity, the lower the fusion cost; the lower the similarity, the greater the fusion cost.

[0365] The fusion can be understood as collapsing two patches into one patch, or merging two patches into one patch. The average angular distance can be pre-calculated or updated in real time during the iteration process.

[0366] Define the i-th patch including the first patch as f i , define the jth patch including the second patch as f j , define the i-th patch f i The normal vector of the jth patch f j The angle between the normal vector and ij , the i-th patch f i With the jth patch f j Angular distance Ang Dist(aij) At least the following formula is satisfied: Ang Dist(aij) =η(1-cosα ij ).

[0367] η is the concave-convex angle parameter.

[0368] In addition to executing the Max-Flow / Min-Cut algorithm, the FFA (Ford-Fulkerson) algorithm, the Goldberg-Tarjan algorithm, etc. may be alternatively implemented to calculate the capacity function or other data with similar properties.

[0369] Taking the maximum flow minimum cut as an example, define the value corresponding to the i-th face f i and the jth patch f j , and the capacity function including the first capacity function is Cap(i,j), the adjustment parameter is defined as w, and the average angular distance is defined as avg(Ang Dist ), then the capacity function Cap(i,j) can at least satisfy the following formula:

[0370] S is the set of traffic source points, and T is the set of traffic sink points.

[0371] If the first capacity function is defined as Cap1, the first differentiation function is E pairwise 1, then Cap1=E pairwise 1∈{E pairwise (x1,x2),Epairwise (x2,x3),......,E pairwise (x i ,x j )}.

[0372] During surface simplification, especially when using the patch probability function within the overall energy function to classify the oral cavity, the following step can be implemented: if the maximum curvature at a vertex falls within a certain range, the vertex is determined to be a seed point. This allows the position and shape of the segmentation line to be determined using the seed point, ensuring that the segmentation line is distributed along the gum line as much as possible, avoiding the gaps between teeth or the alveolar line of the molars.

[0373] Based on this, the model registration in the tooth and jaw recognition method is further described in detail, which may include the following steps.

[0374] Step P31: calling a dental registration model corresponding to the dental model data, and performing point cloud sampling on the dental model data and the dental registration model respectively.

[0375] Define two groups of jaw models as M u and M l For the sake of simplicity, it can be collectively referred to as the basic model M*; the definition corresponds to the dental model M u and M l The two groups of dental registration models are T u and T l For the sake of simplicity, they can be collectively referred to as the registration model T*. The registration model T* can be obtained by matching from a preset model library.

[0376] Based on this, 5000 data points can be sampled and their normal vectors can be calculated respectively in the Origin data table corresponding to the basic model M* and the registration model T*, which lists data in the form of coordinates X, Y, and Z. In this way, the point cloud sampling step is completed.

[0377] Step P32: Calculate the fast point feature histogram features based on the sampling points, and perform a random sampling consensus algorithm on the fast point feature histogram features to obtain coarsely aligned dental and jaw data.

[0378] Based on the sampled data points and normal vector coordinates (point cloud data), the FPFH (Fast Point Feature Histogram) feature can be calculated, and the RANSAC (Random Sample Consensus) algorithm can be implemented on the FPFH feature to obtain a rough transformation relationship between the two point clouds, thereby achieving the technical effect of coarse registration of the sampled point cloud data. It can be understood that the operation object of the coarse registration is the sampled point cloud data, and the coarse degree of registration is relative to the overall dental model data.

[0379] Based on this, the basic model M* or the dental model data can be updated based on the coarse registration result, thereby obtaining the coarsely registered dental data.

[0380] Step P33: performing an iterative closest point algorithm on the coarsely registered dental data.

[0381] Based on the coarse registration of the dental data, the point cloud Origin (X, Y, Z) is finely registered using the ICP (Iterative Closest Point) algorithm to obtain the registered dental model data Matched U and Matched L In one scenario, Matched U Pointing to the maxillary data after registration, Matched L Point to the registered mandibular data.

[0382] For coarse registration, the use of FPFH combined with RANSAC can improve registration accuracy and reduce sampling pressure while ensuring computational efficiency. For the coordination of coarse and fine registration, the use of FPFH combined with ICP can speed up registration and reduce errors.

[0383] The details of the coarse and fine registration processes are too complex to be exhaustive in this disclosure. For example, some implementations of the coarse registration process may also include filtering and denoising the point cloud data and searching for key points using the ISS (Intrinsic Shape Signatures) algorithm. RANSAC can also be replaced with the SAC-IA (Sample Consensus Initial Alignment) algorithm for registration.

[0384] Based on this, the model segmentation in the tooth and jaw recognition method is further described in detail. The model segmentation can be performed by segmenting the tooth model.

[0385] The tooth segmentation model is a neural network model.

[0386] The tooth segmentation model is trained based on a preset dental jaw model and preset tooth labels representing tooth distribution in the preset dental jaw model.

[0387] The preset dental model can be a standard dental model; an actual dental model of a patient or other user; or a dental model determined by an operator based on needs. The preset tooth labels can be tooth labels obtained by manually or machine-verified and marked on any of the above dental models or dental models determined by other methods.

[0388] The dental model is segmented to distinguish the crowns of different tooth positions to assist in the formation of tooth pairs, thereby avoiding excessive reliance on manual labeling for the display of occlusal relationships and the generation of tooth pairs.

[0389] The model segmentation process can be achieved by building a neural network model.

[0390] On the one hand, the neural network model mentioned below can be used as a part of the tooth-jaw relationship recognition model. On the other hand, the neural network model mentioned below can also be distinguished from the tooth-jaw relationship recognition model and exist independently.

[0391] In one embodiment, the model segmentation process in the tooth and jaw recognition method specifically includes the following steps.

[0392] Step P4: call the tooth segmentation model, segment the dental model data in units of tooth positions, and obtain tooth position data.

[0393] In one embodiment, the tooth segmentation model includes a deep learning segmentation module and / or a canary module.

[0394] The deep learning segmentation module is used to generate fine-grained segmentation output.

[0395] The canary module is used to evaluate whether the segmentation output meets the requirements (especially clinical requirements).

[0396] During the segmentation process, the tooth segmentation model converts mesh data into point cloud data for segmentation, and combines the point cloud data with its geometric information (such as patch normal vectors and patch shape). This results in better segmentation results. The re-verification process based on the canary module can correct erroneous outputs in a timely manner.

[0397] The deep learning segmentation module (Deep Learning Segmentation Module) may include a preprocessing module (Data Preprocessing), a segmentation module (Deep Learning Segmentation) and a boundary smoothing module (Boundary Smoothing).

[0398] The Canary Module may include a Confidence Evaluation Module and an Auto-Correction Module.

[0399] The deep learning segmentation module uses point clouds as input and output, and uses kNN (K-Nearest Neighbor, K nearest neighbor classification algorithm) to map the segmentation results back to the neural network side of the deep learning segmentation module.

[0400] The deep learning segmentation module may perform at least one of the following steps when implementing its function:

[0401] Convert point cloud space;

[0402] Execute EdgeConv (edge ​​convolution, also known as DGCNN, Dynamic Graph CNN for Learning on Point Clouds) to calculate vertex edge features and use 2D convolutional layers to aggregate the vertex edge features;

[0403] The output of EdgeConv is concatenated using the Mean-Pooling layer and the Max-Pooling layer to form a global feature descriptor.

[0404] The one-hot encoded classification vector is input to the deep learning segmentation module; several 2D convolutional layers that aggregate the outputs from EdgeConv and the global feature descriptor are stacked to generate classification labels corresponding to tooth positions.

[0405] In addition to using the tooth segmentation model with the special architecture described above, in some embodiments, traditional CNN can also be used.

[0406] Based on this, the data sampling in the tooth and jaw recognition method is further described in detail.

[0407] Taking the two groups of dental model data including the first dental model data and the second dental model data as an example, the data sampling in the dental model recognition method specifically includes the following steps.

[0408] Step P51: determine the tooth position number, surface center point and surface normal vector of the teeth in the dental model data, and combine them to form a plurality of tooth marking data.

[0409] The tooth position numbering may be in the form of the numbering shown in FIG4 , or may be numbered based on FDI.

[0410] The “determination” may include at least one of two aspects.

[0411] First, teeth 6, 7, and 8 are shielded. Specifically, teeth 16 through 18 (t16, t18), teeth 26 through 28 (t26, t28), teeth 36 through 38 (t38), and teeth 46 through 48 (t48). Because these teeth are more likely to be worn or missing, omitting data from these locations not only reduces the data volume but also improves processing efficiency.

[0412] Secondly, the center points of the upper part of the tooth patch and the normal vectors of the patch can be randomly sampled to avoid wasting resources by sampling the upper part of the tooth patch.

[0413] Define the tooth position number as i, then the tooth marking data can be defined as F i , the patch normal vector can be defined as Normal i .

[0414] Step P52: performing a mean removal process on the coordinate portion of the tooth mark data to update the tooth mark data.

[0415] When the patch normal vector i When saved in coordinate form, the "coordinate part of the tooth mark data" includes the face normal vector Normal i and the coordinates of the center point of the patch; otherwise, the "coordinate part in the tooth marking data" only includes the coordinates of the center point of the patch. Corresponding to the tooth position number i, the coordinate part is defined as Coords i The de-meaning process can be interpreted as: Coords i '=Coords i -mean(Coords i ).

[0416] After the above processing, the differences or fluctuations between the elements in the tooth marking data will be more significant, and the data obtained by calculation will have higher discrimination, which is more conducive to subsequent judgment.

[0417] Step P53: performing tooth position matching on the tooth marking data corresponding to the first jaw model data and the tooth marking data corresponding to the second jaw model data according to the tooth position number to obtain tooth pair data.

[0418] The output tooth pair data may be presented in the form of a sequence, which may sequentially include several tooth pair data groups (eg, Pair 12-42), each of which includes at least two pieces of tooth data.

[0419] The pairing method is determined based on the needs of jaw identification. For example, teeth with the same name in the upper jaws of different jaws (for example, both teeth are numbered t11) can be combined to form a tooth pair. Alternatively, teeth with central symmetry in the lower jaws of the same jaw (for example, numbered t11 and numbered t41) can be combined to form a tooth pair. In this embodiment, using teeth that have an occlusal relationship as a pair, this pairing method can more accurately distinguish between the upper and lower jaws.

[0420] The process of constructing tooth marker data can be completed in a four-quadrant coordinate system to lay the foundation for the tooth matching process required for jaw recognition. Therefore, in one embodiment, the jaw recognition method may include the following steps.

[0421] Step P511: construct a jaw coordinate system, set the first jaw model data in the first and second quadrants, and set the second jaw model data that forms an occlusal relationship with the first jaw model data in the third and fourth quadrants.

[0422] The process of setting the first and second jaw model data can be interpreted as placing the model corresponding to the first and second jaw model data in the corresponding quadrants. The first, second, third, and fourth quadrants are the same or similar to the first, second, third, and fourth quadrants described above. When setting the jaw model corresponding to the first and second jaw model data, the jaw model corresponding to the second jaw model data is set as shown in Figure 4 without losing the occlusal relationship between the two.

[0423] Step P512: Determine the tooth position numbers, surface center points, and surface normal vectors of teeth 1, 2, 3, 4, and 5 located in the four quadrants in the dental model data, and combine them to form a plurality of tooth marking data.

[0424] Taking the first quadrant as an example, the eleventh tooth t11 to the fifteenth tooth t15 are sampled and screened, and their tooth position numbers and other data are determined, and then the tooth marking data F is formed. 11 、F 12 、F 13 、F14 、F 15 It can be seen that in this embodiment, under ideal conditions, each quadrant can form five sets of tooth marking data, and the four quadrants can form a total of 20 sets of tooth marking data. In special cases such as missing teeth, the number of tooth marking data will be reduced.

[0425] Step P53 may include the steps of pairing the tooth data corresponding to the teeth with the same number located in the first quadrant and the third quadrant, and pairing the teeth corresponding to the teeth with the same number located in the second quadrant and the fourth quadrant, to obtain the tooth pair data.

[0426] "Teeth with the same number" can be interpreted as teeth with the same second position based on the FDI tooth position representation. In this way, the teeth with the same number corresponding to the first and fourth quadrants can form five groups of tooth pair data. 1j-4j , the teeth with the same sign in the second and third quadrants can form five groups of tooth pair data 2j-3j . j=1,2,3,4,5.

[0427] The connector that defines the pairing operation is The above operation can be further expanded as follows:

[0428] In one embodiment, the jaw recognition method includes the steps of obtaining a numerical form of the relationship identification information, and determining upper and lower jaw determination information of the first jaw model data and the second jaw model data according to the numerical relationship.

[0429] The numerical relationship can be the relationship between the numerical relationship identification information and the set threshold, or can be the discreteness between multiple numerical relationship identification information. The numerical relationship determination can also include a step of eliminating abnormal values.

[0430] The present invention does not limit the relationship identification information to be in the form of a numerical value. Labels such as "yes" and "no" that can be converted into numerical values ​​based on preset rules can also be used as the content of the relationship identification information.

[0431] For example, the first tooth on top and the second tooth on bottom can be assigned the first label, the first tooth on bottom and the second tooth on top can be assigned the second label, both teeth on top can be assigned the third label, and both teeth on bottom can be assigned the fourth label. Compared to assigning labels A for the first tooth on top and B for the second tooth on bottom, and C for the second tooth on top and D for the second tooth on bottom, the four positional relationships can be assigned labels AC, AD, BC, and BD, for a total of eight labels. This can reduce the number of labels involved in the calculation and speed up training and prediction.

[0432] In a second embodiment provided by the present invention, as shown in FIG8 , the jaw identification method includes the following steps.

[0433] Step S3A: in the tooth pair data, when the tooth to be measured in the first jaw model points to the maxillary tooth and the tooth to be measured in the second jaw model data points to the mandibular tooth, obtain relationship identification information in a numerical form of a first label value.

[0434] Step S3B: in the tooth pair data, when the tooth to be measured in the first jaw model points to the mandibular tooth and the tooth to be measured in the second jaw model data points to the maxillary tooth, obtain relationship identification information in a numerical form of a second label value.

[0435] The tooth data pointing to the upper jaw and the tooth data pointing to the lower jaw can be specifically interpreted as: the tooth position or tooth represented by the tooth data is located in the upper jaw, or the tooth position or tooth represented by the tooth data is located in the lower jaw.

[0436] Steps S3A and S3B illustrate two ways of obtaining relationship identification information, indicating that relationship identification information with different tag values ​​is provided in different situations. It is worth noting that there is no necessary order or dependency between the two steps.

[0437] In one embodiment, the tooth and jaw recognition method specifically includes the steps of: determining the upper and lower jaw determination information based on the numerical label values ​​corresponding to the multiple groups of tooth pairs and according to the voting score results.

[0438] In one case, different tooth pairs may provide different relationship identification information. When the relationship identification information indicates the relative position of the tooth pairs, or when the jaw information includes maxillary and lower jaw determination information, this may lead to disagreements in the maxillary and lower jaw determinations of the jaw model. In this case, the maxillary and lower jaw determination information can be determined based on a "majority rule" voting and scoring.

[0439] The upper and lower jaw determination information may include upper jaw determination information or lower jaw determination information for the first dental jaw model, or upper jaw determination information or lower jaw determination information for the second dental jaw model.

[0440] In one embodiment, the tooth and jaw identification method specifically includes the following steps.

[0441] Step S31, calculating the mean of the label values ​​corresponding to the numerical form of multiple groups of tooth pair data;

[0442] Step S32: determining the upper and lower jaw determination information of the two sets of dental model data according to the numerical relationship between the label value mean and the prediction threshold.

[0443] The data sample foundation for identifying upper and lower jaw information on dental models has been expanded to include relationship identification information corresponding to multiple tooth pairs. By calculating the mean of the numerical label values ​​for judgment, overall stability can be increased, preventing individual errors from directly and excessively affecting the judgment results.

[0444] The mean does not necessarily refer to the arithmetic mean. Different weights can also be assigned to different tooth pairs in different regions to calculate a weighted mean. For example, a larger weight can be assigned to the front teeth area, where the occlusion relationship is clearer and the teeth are more neatly arranged, and a smaller weight can be assigned to the back teeth area.

[0445] In order to compress the data volume to a greater extent, compared with setting labels for each tooth position, the present invention sets labels for each position relationship combination of tooth pairs, thereby achieving the effect of simplifying calculations.

[0446] The above embodiment can be combined with the above steps S3A and S3B. In this way, different label values ​​can be set for different combinations of tooth pair data pointing to jaws, thereby comprehensively and quickly determining the relative positional relationship between two sets of jaw model data.

[0447] The numerical values ​​between the label value mean and the prediction threshold may be compared. Specifically, when the label value mean is greater than the prediction threshold, the first jaw model data may be determined to be pointing to the maxilla, or when the label value mean is less than the prediction threshold, the first jaw model data may be determined to be pointing to the maxilla.

[0448] Based on the above idea, the second label value is set to be greater than the first label value, that is, the label value when the first tooth is on top and the second tooth is on the bottom is smaller than the label value when the second tooth is on top and the first tooth is on the bottom. The jaw recognition method can then include the following steps.

[0449] Determine whether the mean of the label values ​​is greater than the prediction threshold;

[0450] If it is greater than, it is determined that the first jaw model data points to the mandible, and the second jaw model data points to the maxilla;

[0451] If it is less than, it is determined that the first jaw model data points to the upper jaw, and the second jaw model data points to the lower jaw.

[0452] In this way, the positional relationship between the corresponding jaws can be inferred based on the combination of the positional relationships of several teeth in the tooth pair and the corresponding relationship between teeth and jaws. The determination of these positional relationships is visualized as numerical calculations and size judgments, achieving higher accuracy and automation.

[0453] The setting of the prediction threshold can be related to whether the jaws pointed to by the two sets of jaw model data are missing teeth. If teeth are missing, the prediction threshold can be set higher, and if teeth are not missing, the prediction threshold can be set lower. Based on this, the jaw recognition method can include the steps of: determining whether the jaw model data (i.e., the first jaw model data and the second jaw model data) are missing teeth; if teeth are missing, setting the prediction threshold to a higher first threshold; if teeth are not missing, setting the prediction threshold to a lower second threshold.

[0454] For example, if the second label value is set to 1, the first label value is set to 0. When no tooth is missing, the predicted relationship identification information can be in the form of a 10-dimensional vector. In this case, the prediction threshold can be set to 0.25; in one embodiment, the mean of the label values ​​is the arithmetic mean. When tooth is missing, the predicted relationship identification information is less than 10 dimensions. In this case, the prediction threshold can be set to 0.5; in one embodiment, the mean of the label values ​​is the arithmetic mean.

[0455] In summary, the jaw recognition method provided by the present invention uses the relational identification information that characterizes the relative relationship of the tooth pairs to determine the jaw information of different jaws where the tooth pairs are located, which is equivalent to providing tooth pairs as a reference for the jaw recognition process, making the overall recognition process more robust; and, since the tooth recognition scheme is relatively mature, compared with schemes that require excessive manual intervention and include complex preprocessing steps, it has the characteristics of both high efficiency and stability; since the tooth pairs are discrete, it can also avoid individual errors affecting the overall results, and can adapt to recognition in a variety of complex scenarios.

[0456] In clinical dental practice, the accuracy of the occlusal relationship between the maxillary and mandibular digital models is crucial, directly impacting both the tooth arrangement and the final correction outcome. If the occlusal relationship between the maxillary and mandibular digital models is inconsistent with the user's actual occlusal relationship, this can lead to deviations from the intended treatment outcome. Therefore, optimization and adjustment of the occlusal relationship between the maxillary and mandibular models is often necessary in clinical practice.

[0457] In related technologies, the occlusion adjustment of the upper and lower jaw models mainly relies on manual labor, which is performed manually through third-party editing tools to optimize and adjust the upper and lower jaw models. This requires manual participation, has a high technical threshold, and the occlusion adjustment results are greatly affected by human factors.

[0458] With the development of intelligent oral processing technology, oral rehabilitation technology based on 3D digital models has gained widespread clinical application. Taking orthodontics as an example, 3D modeling technologies such as oral scanning can be used to construct digital models of the patient's upper and lower jaws, and tooth arrangement and correction plans can be implemented based on these digital models.

[0459] In the field of orthodontics and prosthodontics, the accuracy of the occlusal relationship between the maxillary and mandibular models is crucial. If the occlusal relationship of the maxillary and mandibular digital models is inconsistent with the patient's actual occlusal relationship, it can lead to deviations from the intended treatment outcome. For example, this can affect the comfort of the final designed restoration within the patient's mouth, resulting in the restoration not functioning properly.

[0460] In dental clinical applications, an intraoral scanner is typically used to convert the user's actual intraoral maxillary and mandibular information into a three-dimensional digital model. This 3D digital model primarily includes the morphology of the teeth and gums. Furthermore, the intraoral scanner can directly capture the user's oral occlusion. Generally, after scanning and acquiring the maxillary and mandibular models, the mouth is left in a natural occlusion state. The intraoral scanner is then used to scan the entire jaw area in this occlusion state. Based on their respective overlapping relationships with the entire jaw, the upper and lower jaws are aligned to the current occlusion state, resulting in an occlusion relationship closer to reality.

[0461] However, due to deformation errors during the scanning process (such as model deformation errors of the mandibular and maxillary data themselves, deformation errors of the full jaw data), as well as the influence of interference factors such as bubble noise on the model, the occlusal relationship of the scanned mandibular and maxillary models may be inaccurate. Moreover, in actual clinical practice, as the correction stage progresses, it is often necessary to provide oral digital models with different occlusal relationships according to clinical needs. This requires adjusting the occlusal relationship of the mandibular and maxillary models. The operation process of adjusting the occlusal relationship of the mandibular and maxillary models is called occlusal adjustment.

[0462] One possible implementation method is to make occlusal adjustments for the occlusal relationship of the upper and lower jaw models, which can rely on manual and third-party editing tools. For example, manual adjustments can be made based on the user's intraoral photos and the original upper and lower jaw models. Specifically, the operator refers to the user's occlusal photos in multiple directions, and first makes preliminary adjustments to the upper and lower jaw model files through operations such as translation and rotation on the third-party model editor software, and then makes fine adjustments based on the midline position, posterior tooth coverage, etc., to achieve the purpose of occlusion consistency with the photo. For example, the occlusal adjustment of the upper and lower jaw models can also be made according to the occlusal adjustment algorithm. Specifically, the operator can pre-mark the tooth areas of the upper and lower jaw models, and then set the iterative parameters to iteratively calculate the distance between the maxillary and mandibular tooth pairs, and adjust the occlusal relationship until the calculated distance meets the preset conditions to obtain a more accurate occlusal result.

[0463] In the above-mentioned related technical solutions, the process of occlusal adjustment requires different degrees of manual intervention, requiring the operator to have extremely rich experience in oral restoration and the ability to perceive the difference between two-dimensional photos and three-dimensional models. The adjustment results obtained by operators of different levels vary greatly. Even for the same operator on the same patient, the results of occlusal adjustment cannot be completely consistent in different scenarios and working conditions. In addition, the manual setting of parameters also needs to rely on the operator's experience, and there is a lack of objective guidance on the parameter rules for complex cases, resulting in low efficiency and high subjective dependence of occlusal adjustment.

[0464] Furthermore, the above-mentioned occlusion adjustment process inevitably relies on third-party model editing software. The operator needs to perform various operations on the model in the software, such as translation, rotation, and parameter configuration. This requires the operator to be proficient in the editing software, which increases the learning cost for the operator.

[0465] Based on the above problems, the embodiments of this specification provide an occlusion adjustment method, device, electronic device, storage medium and computer program product, aiming to provide an occlusion adjustment method that reduces dependence on manual experience and third-party editing software, and performs occlusion adjustment based on the alignment of the original model and the occlusion relationship model, which helps to improve the efficiency of occlusion adjustment and the accuracy of the adjustment results.

[0466] In some embodiments, this specification provides a method for occlusal adjustment, which can be performed by an electronic device. In the embodiments of this specification, the electronic device can be any type of device suitable for implementation, such as an intraoral scanner, a computer, a server, a mobile terminal, a wearable device, etc., and this specification does not limit this.

[0467] As shown in FIG10 , in some embodiments, the occlusal adjustment method exemplified in this specification includes:

[0468] S1010: Obtain a first digital model and a second digital model of the user's oral cavity.

[0469] In the embodiments of this specification, the oral cavity refers to the cavity within a user's mouth, which may include tissues or organs such as teeth, dental crowns, gums, and jawbone. For example, when performing oral restoration on a user, it is necessary to convert the user's oral information into a three-dimensional digital model. This three-dimensional digital model is the first digital model of the user's oral cavity as described in this specification.

[0470] In some embodiments, a first digital model may be obtained by performing a three-dimensional digital scan of the user's oral cavity. The first digital model includes at least a tooth model of the user's oral cavity and may further include a model of the gums or other areas.

[0471] It is worth noting that in actual oral restoration scenarios, only a partial occlusal restoration of the user's mouth may be required, so the first digital model may only include the maxillary or mandibular model requiring restoration. Alternatively, a full occlusal restoration of the user's mouth may be required, so the first digital model may include both the maxillary and mandibular models. This is readily understood by those skilled in the art, and will not be further explained in this specification unless otherwise required.

[0472] In some embodiments, an oral cavity scanning device may be used to scan the user's oral cavity to construct a first digital model of the user's oral cavity, wherein the first digital model is constructed by scanning the user's oral cavity without limiting the user's oral cavity.

[0473] In one example, when scanning the user's mouth, a two-dimensional color image and a three-dimensional point cloud data frame can be obtained for each scanning position. Then, the two-dimensional color image is recognized based on a preset artificial intelligence model to obtain the recognition results corresponding to the pixel points on the two-dimensional color image. Then, for the same scanning position, the recognition results are mapped to the three-dimensional data points of the three-dimensional point cloud data frame. Finally, all the three-dimensional point cloud data frames are fused to obtain the first digital model of the user's mouth.

[0474] It is understood that the above process is only an example of obtaining the first digital model in the embodiments of this specification, and the process of obtaining the first digital model is not limited to the above method. The first digital model can also be constructed and obtained by other methods with reference to relevant technologies in this field, and this specification will not elaborate on this.

[0475] In the above embodiment, the occlusal relationship of the original first digital model is not directly adjusted. Instead, a second digital model of the user's mouth is further introduced. The second digital model can reflect the actual occlusal relationship of the user's mouth under a preset occlusal state. The occlusal relationship of the first digital model is adjusted based on the second digital model, and an adjustment result that is closer to the user's actual occlusal relationship can be obtained.

[0476] In the embodiments of this specification, the second digital model refers to a three-dimensional digital model of the user's mouth in an occlusal state. The occlusal state refers to any one or more occlusal states required for oral rehabilitation. In general, the occlusal state can be the state of the user's upper and lower jaws in natural occlusion. In certain special scenarios, it may be necessary to align the user's occlusal relationship in a specific occlusal position, such as an overbite, deep overjet, or open bite, so that the user can occlude according to the specific occlusal position.

[0477] In the embodiments of this specification, the second digital model may only include a digital model of the target mouth in a specific occlusal state. Thus, based on the occlusal adjustment method of this specification, the occlusal relationship of the first digital model can be adjusted based on the second digital model. The second digital model may also include digital models of the user's mouth in multiple occlusal states. Thus, based on the occlusal adjustment method of this specification, the occlusal relationship of the first digital model can be adjusted based on the second digital model in each occlusal state, thereby obtaining adjustment results for the first digital model in multiple occlusal states.

[0478] It should be noted that the occlusion adjustment method between the second digital model and the first digital model in each occlusal state can refer to each other. The occlusal adjustment method of this specification is described below using the second digital model in one occlusal state as an example.

[0479] In some embodiments, the process of obtaining the second digital model may include: having the user bite a specific material (e.g., an impression) according to a preset occlusal state (e.g., a natural occlusal state), and obtaining an occlusal result after the impression material solidifies. Then, pouring a casting material (e.g., plaster) into the impression occlusal result to fill the upper and lower jaw areas respectively, and removing the casting material after solidification to obtain the upper and lower jaw dentition models. Finally, the dentition model is scanned by digital means (e.g., blue light scanning or other scanning methods) to obtain a second digital model containing the correct occlusal relationship.

[0480] Of course, it is understood that the above process is only an example of obtaining the second digital model in the embodiments of this specification, and the process of obtaining the second digital model is not limited to the above method. Those skilled in the art can also construct a second digital model of the user's mouth by other methods, such as oral CBCT (Cone Beam Computer Tomography, oral and maxillofacial cone beam scanning), model reconstruction based on oral two-dimensional images, etc., with reference to relevant technologies, and this specification will not go into details.

[0481] In the field of computer graphics, mesh models are often used to represent the geometric form of three-dimensional objects. In the embodiments of this specification, the model forms corresponding to the first digital model and the second digital model can be mesh models.

[0482] The mesh model is composed of a large number of discrete vertices, edges connecting two vertices, and mesh patches composed of multiple edges. In some embodiments, the mesh patches can be triangular patches, that is, the face of a triangle composed of three vertices is a mesh patch, so that the mesh model is composed of a continuous surface composed of a large number of triangular patches. In other embodiments, the mesh patches can be quadrilaterals, that is, the face of a quadrilateral composed of four vertices is a mesh patch, so that the mesh model is composed of a continuous surface composed of a large number of quadrilateral patches.

[0483] The point cloud data mentioned below in this specification may be a data set consisting of all vertex-related information on a mesh model.

[0484] S1020: Perform model registration on the first digital model and the second digital model to obtain a spatial transformation relationship for mapping the first digital model to the second digital model.

[0485] As can be seen from the above, the goal of occlusal adjustment is to restore the correct occlusal relationship contained in the second digital model to the first digital model. In other words, the occlusal relationship of the first digital model is adjusted based on the occlusal relationship of the second digital model, so that the upper and lower jaw occlusal relationship of the first digital model is closer to the user's actual occlusal relationship.

[0486] In the embodiments of this specification, after obtaining the first and second digital models, a mapping relationship between points on the first digital model and points on the second digital model can be determined. This mapping relationship is referred to as a spatial transformation relationship. Therefore, based on this spatial transformation relationship, the spatial position of the first digital model can be adjusted to match or approximate the user's actual occlusion relationship.

[0487] In the implementation manner of this specification, the model alignment of two digital models refers to the alignment of the tooth area included in the model. It can be understood that the three-dimensional digital model obtained by digital scanning and other means, in addition to the tooth area, often also includes noise parts such as gums and bases. These noise parts may cause a decrease in accuracy during model alignment and affect the occlusal adjustment results.

[0488] In some embodiments, the first digital model and the second digital model may be digital models of the dental region of the user's mouth. For example, a digital scanning method may be used to obtain an original oral model of the user's mouth. The dental region of the oral model is then segmented, retaining only the dental region to obtain the first digital model. The segmentation process for the second digital model is similar, as described below in conjunction with FIG11 .

[0489] In some embodiments, as can be seen from the foregoing, both the first and second digital models are mesh models, and their corresponding model data is point cloud data. When the mesh model is highly accurate, the amount of vertex data corresponding to the model is enormous, resulting in a large amount of computation during the model registration process. This can easily lead to local computational convergence and local minimization, resulting in unsatisfactory registration results.

[0490] In some embodiments of this specification, the point cloud data of the first digital model and the second digital model can be downsampled before model registration, thereby reducing the number of vertices and reducing computational overhead. The point cloud downsampling process will be described below in conjunction with Figure 13.

[0491] It is understandable that the above-mentioned processes of model segmentation of the tooth area and point cloud downsampling are optional and not necessary.

[0492] In some embodiments, a model registration algorithm based on point cloud data can be used to align the first digital model and the second digital model, such as a PCA (Principal Component Analysis) algorithm, a SICP (Sparse Iterative Closest Point) algorithm, etc., which is not limited in this specification.

[0493] Through the model registration process described above, a spatial transformation relationship between the first and second digital models is obtained. This spatial transformation relationship can be understood as a mapping relationship between points on the first digital model and points on the second digital model. Based on this spatial transformation relationship, any point on the first digital model can be mapped to the positional space of the second digital model, thereby adjusting the fit of the first digital model.

[0494] In some embodiments, in their initial state, the spatial positions and orientations of the first and second digital models may differ significantly. For example, the models may be flipped left-to-right, up-to-down, front-to-back, or a combination of these. Directly performing model registration may cause the calculation process to fall into a local optimum, preventing the models from correctly converging to overlap.

[0495] In some embodiments of this specification, during the registration process of a first digital model and a second digital model, the principal axis direction of the first digital model can be calculated. Then, through model registration, the first digital model can be flipped so that the principal axis direction is roughly aligned with that of the second digital model, completing a coarse registration. Subsequently, fine registration is performed based on the coarsely registered first and second digital models, effectively reducing the risk of registration failure and improving occlusal adjustment accuracy. This model registration process will be described below with reference to Figures 15 and 16.

[0496] S1030: Adjust the occlusal relationship of the first digital model based on the spatial transformation relationship to obtain a target digital model of the user's oral cavity.

[0497] As can be seen from the foregoing, the spatial transformation relationship represents the spatial mapping relationship between the first and second digital models. Therefore, when performing an articulation adjustment on the first digital model, each point on the first digital model can be mapped and adjusted based on this spatial transformation relationship. For example, each point on the first digital model can be spatially transformed based on the spatial transformation relationship to obtain a target digital model with the same articulation state.

[0498] Based on the above, it can be seen that in the implementation mode of this specification, the original first digital model is aligned based on the second digital model containing the occlusal state, and the occlusal adjustment of the first digital model is performed based on the spatial conversion relationship obtained by the alignment, so that the real occlusal relationship represented by the second digital model is restored to the first digital model, which can effectively improve the accuracy of the occlusal adjustment result and be more consistent with the user's real occlusal relationship.

[0499] In addition, in the implementation mode of this specification, the operator only needs to generate the first digital model and the second digital model of the user's mouth as needed, and then input the two sets of models into the algorithm end to automatically complete the occlusion adjustment process. There is no need for manual editing, nor is there any need to manually adjust the relevant parameters of the occlusion adjustment process, thereby avoiding occlusion adjustment deviations caused by subjective factors, reducing dependence on third-party model editing tools, simplifying the occlusion adjustment algorithm process, and improving algorithm efficiency.

[0500] Furthermore, in the embodiments of this specification, the occlusion adjustment method is more flexible, and can be performed on both half-mouth digital models and full-mouth digital models. It can also adapt to adjustment scenarios involving multiple occlusal states. For example, if the occlusal relationship of a first digital model needs to be adjusted under multiple occlusal states, it is only necessary to obtain the same first digital model, then obtain multiple second digital models under different occlusal states, and repeat the above method process in sequence. This allows occlusal adjustment to be performed on the first digital model under multiple occlusal states, thus meeting diverse oral rehabilitation scenarios.

[0501] In combination with the foregoing, it can be seen that in some embodiments, the first digital model and the second digital model described in this specification are both models obtained by segmenting the tooth area of ​​the oral model, that is, the first digital model and the second digital model only include the tooth area of ​​the user's mouth, so that in the model alignment process, alignment is only based on the tooth model, avoiding irrelevant noise interference and improving the model alignment accuracy. This is explained below in conjunction with Figure 11.

[0502] As shown in FIG11 , in some embodiments, the occlusion adjustment method exemplified in this specification, the process of obtaining the first digital model and the second digital model, includes:

[0503] S1110 , digitally process the user's oral cavity to obtain an oral cavity model, and segment the tooth region of the oral cavity model to obtain a first digital model.

[0504] S1120: Digitally process the user's oral cavity in an occlusal state to obtain an occlusal model, and segment the tooth region of the occlusal model to obtain a second digital model.

[0505] In the implementation manner of this specification, the oral model and occlusal model of the user's mouth can first be obtained through various digital processing methods. The oral model refers to a three-dimensional digital model obtained by digitally processing the intraoral information of the user's mouth, which may include models of tissues and organs such as teeth, gums, and cheeks. The occlusal model refers to a three-dimensional digital model of the user's mouth in one or more occlusal states, which may also include models of tissues and organs such as teeth, gums, and cheeks.

[0506] The methods for obtaining oral models and occlusal models have been given above in this specification. Those skilled in the art can understand and fully implement them by referring to the above. For example, intraoral scanning, oral CBCT, model reconstruction based on oral two-dimensional images, etc. can be used, and this specification will not go into details.

[0507] In the implementation manner of this specification, the purpose of model segmentation is to identify the tooth area and non-tooth area from the model. The non-tooth area is, for example, the gum, base, cheek and other noise parts, so that only the model part of the tooth area is retained through model segmentation, and the model part of the non-tooth area is deleted.

[0508] It can be understood that there are many three-dimensional model segmentation algorithms, and the implementation methods of this specification do not impose too many restrictions on this. Any model segmentation algorithm can be used. Model segmentation algorithms include but are not limited to geometric topology, neural networks, deep learning and other methods.

[0509] In one embodiment, a deep learning-based model segmentation algorithm is used as an example. First, a deep learning-based tooth segmentation network can be pre-built. The input of the tooth segmentation network is a 3D model, and the output is the tooth region of the 3D model. The tooth segmentation network can be a deep learning network based on architectures such as 3D-CNNs, PointNet, and PointNet++.

[0510] During the network training phase, a large training dataset containing 3D models of tooth regions can be collected through manual annotation. The tooth segmentation network is trained based on the training dataset until the network converges and training is terminated. The network training process can be performed using a traditional supervised training process, which can be fully understood and implemented by those skilled in the art by referring to relevant techniques, and will not be further elaborated in this specification.

[0511] During the prediction phase, the oral cavity model or occlusion model can be input into the trained tooth segmentation network, which then predicts and outputs the tooth regions corresponding to the model. For example, inputting the oral cavity model into the tooth segmentation network yields a first digital model containing only the tooth regions in the oral cavity model. Inputting the occlusion model into the tooth segmentation network yields a second digital model containing only the tooth regions in the occlusion model.

[0512] In one example, the oral model uses the upper and lower jaw models of a user's mouth as an example. Figure 12 shows the visualization result of processing the oral model using the above-mentioned tooth segmentation network. As shown in Figure 12, the colored areas in the figure are the tooth areas, and the gray areas are the non-tooth areas.

[0513] Continuing with FIG12 , in this example, after segmenting the tooth region from the oral model, each tooth can be further segmented individually, such as each tooth in FIG12 . Of course, the model segmentation result can also be such that only the tooth region and the non-tooth region are segmented, without the need to segment each tooth individually.

[0514] The above description uses the tooth region segmentation process of the oral model as an example. The segmentation process of the occlusal model is the same, and the above description can be referred to, and no further details will be given.

[0515] From the above, it can be seen that in the implementation mode of this specification, the first digital model and the second digital model are obtained by segmenting the oral model and the occlusal model, thereby removing the interference of non-tooth areas such as gums, noise, and base, providing an accurate data basis for subsequent model alignment, and thus improving the algorithm accuracy of occlusal adjustment.

[0516] As can be seen from the foregoing, both the first and second digital models are mesh models, and their corresponding mesh vertex data is point cloud data. When the mesh model is highly accurate, the corresponding vertex data volume is enormous, resulting in a large computational load during the model registration process. This can easily lead to local computational convergence and local minimization, resulting in unsatisfactory registration results.

[0517] Therefore, in some implementations of this specification, the point cloud data of the first digital model and the second digital model may be downsampled before model registration. This process is described below with reference to FIG13 .

[0518] As shown in FIG13 , in some embodiments, the occlusion adjustment method exemplified in this specification, based on the process of performing model registration on the first digital model and the second digital model, includes:

[0519] S1310: Acquire first point cloud data corresponding to the first digital model and second point cloud data corresponding to the second digital model.

[0520] S1320: Downsample the first point cloud data and the second point cloud data, and perform point cloud registration based on the downsampled first point cloud data and the second point cloud data to obtain a space transformation matrix.

[0521] Based on the above, it can be seen that the data corresponding to the first digital model and the second digital model are point cloud data, so the point cloud data of the first digital model is defined as first point cloud data, and the point cloud data corresponding to the second digital model is defined as second point cloud data.

[0522] The purpose of point cloud downsampling is to reduce the number of vertices in the point cloud model. There are many algorithms for downsampling the first point cloud data and the second point cloud data, such as grid-based point cloud downsampling algorithms, Poisson equation-based point cloud downsampling algorithms, etc. This manual does not limit this.

[0523] Taking the grid-based point cloud downsampling algorithm as an example, the point cloud data can be spatially divided into grids of fixed size. The grid is a cube with a certain side length, or it can be other shapes, which are not limited here. It can be understood that by using the grid to spatially divide the point cloud data, for a grid with points, each grid will contain one or more points. Then, the distance between each point in the grid and the center point of the grid can be calculated separately. After that, only the point with the smallest distance is retained, and the remaining points in the grid are deleted. By performing the above operation on each grid, the point cloud downsampling process can be completed.

[0524] It can be understood that after processing the first point cloud data and the second point cloud data through the above process, the number of vertices contained in the point cloud data will be greatly reduced, thereby reducing the data volume and computing overhead. Figure 14 shows the visualization result after downsampling the first point cloud data.

[0525] In the embodiments of this specification, after downsampling the first and second point cloud data, point cloud registration can be performed based on the reduced data volume of the first and second point cloud data to obtain a spatial transformation relationship between the first and second digital models. The point cloud registration process is described below with reference to FIG15 .

[0526] As shown in FIG. 15 , in some embodiments, the occlusion adjustment method exemplified in this specification, based on the process of performing model registration on the first digital model and the second digital model to obtain a spatial transformation relationship, includes:

[0527] S1510: Acquire first point cloud data corresponding to the first digital model and second point cloud data corresponding to the second digital model.

[0528] As can be seen from the foregoing, in the embodiments of this specification, the point cloud data of the first digital model is defined as the first point cloud data, and the point cloud data corresponding to the second digital model is defined as the second point cloud data. When performing point cloud registration, the first point cloud data can be used as the source point cloud data, and the second point cloud data can be used as the target point cloud data.

[0529] S1520 , spatially flipping the principal axis of the first point cloud data to obtain candidate point cloud data in multiple principal axis directions.

[0530] It can be understood that the principal axis direction of the point cloud refers to the direction of the principal eigenvector of the point cloud centroid. The principal axis direction of the point cloud can be calculated through the following process: first, the centroid of the point cloud can be calculated. The centroid is the average position of all points in the point cloud data. The purpose of calculating the centroid is to convert the point cloud into a coordinate system centered on the centroid; then, for each point in the point cloud, the coordinates of the point relative to the centroid are calculated respectively to construct a covariance matrix. The covariance matrix can reflect the correlation between each point in the point cloud data and the centroid; then, the eigenvectors and eigenvalues ​​of multiple main directions of the point cloud data are calculated according to the covariance matrix, and finally the eigenvalues ​​of these eigenvectors are sorted. The eigenvector with the largest eigenvalue represents the principal eigenvector of the point cloud data, and the direction of the principal eigenvector is the principal axis direction of the point cloud.

[0531] In the implementation manner of this specification, after the first point cloud data and the second point cloud data are acquired, the principal eigenvectors and principal axis directions of the first point cloud data and the second point cloud data may be calculated by the aforementioned method and process.

[0532] It is understood that if the principal axis directions of the first point cloud data and the second point cloud data are consistent or very close, the spatial positional relationship between the first point cloud data and the second point cloud data should also be consistent or very close. Conversely, if the principal axis directions of the first point cloud data and the second point cloud data are significantly different, such as when they are flipped left-right, up-down, front-to-back, or a combination of flips, the spatial positional relationship between the first point cloud data and the second point cloud data will also be significantly different. Direct registration may cause the calculation process to fall into a local optimal solution, and the model may not correctly converge to overlap.

[0533] Based on this, in some embodiments of this specification, the principal axis of the first point cloud data is traversed in all directions by flipping the principal axis to obtain candidate point cloud data in multiple principal axis directions, which is explained below in conjunction with Figure 16.

[0534] As shown in FIG. 16 , in some embodiments, the process of obtaining candidate point cloud data in multiple principal axis directions in the occlusion adjustment method exemplified in this specification includes:

[0535] S1521. Process the first point cloud data to obtain a first eigenvector of the centroid of the first point cloud data.

[0536] S1522 : Flip the first point cloud data based on the principal axis direction represented by the first eigenvector to obtain candidate point cloud data under multiple principal axis directions.

[0537] In conjunction with the foregoing, it can be seen that the centroid of the first point cloud data can first be calculated. Then, the coordinates of each point in the first point cloud data relative to the centroid can be calculated and a covariance matrix can be constructed. Then, the principal eigenvector of the first point cloud data can be calculated based on the covariance matrix. This principal eigenvector is also the first eigenvector described in this specification, and the direction of the first eigenvector is the principal axis direction of the first point cloud data. The specific process of calculating the first eigenvector and the principal axis direction will be understood by those skilled in the art in conjunction with the foregoing and will not be described in detail here.

[0538] Specifically, the right-hand rule is a method used to determine the axis, which ensures the directional consistency of the coordinate system. Its specific principle is: when you need to determine the direction of the coordinate axis in a three-dimensional space, you can use your index finger and middle finger to point to the x-axis and y-axis respectively, then the direction pointed by your thumb is the z-axis direction.

[0539] According to the right-hand rule above, for a three-dimensional eigenvector, determining the directions of two dimensions allows the third dimension to be calculated through a cross product. Therefore, there are only four possible combinations of directions for the first eigenvector: [1,1], [1,-1], [-1,1], and [-1,-1], where 1 indicates the current axis remains unchanged and -1 indicates it is flipped. The direction of the third axis can be calculated through a cross product, eliminating the need for repeated calculations.

[0540] Therefore, in the implementation of this specification, based on the above four possibilities, the direction of the first eigenvector can be changed in sequence, that is, the main axis direction of the first point cloud data can be changed, so as to obtain candidate point cloud data corresponding to the four coordinate axis directions.

[0541] S1530 , performing point cloud registration based on principal component analysis on each candidate point cloud data and the second point cloud data, to obtain a similarity and a transformation matrix corresponding to each candidate point cloud data.

[0542] The PCA algorithm, also known as the principal component analysis algorithm, is a statistical method. In some embodiments of the present disclosure, the PCA algorithm may be used to achieve registration between the candidate point cloud data and the second point cloud data.

[0543] Combined with the above, it can be seen that the candidate point cloud data refers to multiple point cloud data obtained by flipping the first point cloud data in the main axis direction. The purpose of PCA alignment is to align each candidate point cloud data with the second point cloud data respectively, and then find the point cloud that is most similar to the second point cloud data, that is, the target candidate point cloud data.

[0544] Taking any candidate point cloud data as an example, when performing PCA registration on the candidate point cloud data and the second point cloud data, the principal components of the two point cloud data can be aligned first. The principal component is the first eigenvector mentioned above. That is, the first eigenvector of the candidate point cloud data is aligned with the principal eigenvector of the second point cloud data to obtain the rotation matrix and center of mass translation matrix between the two, and the transformation matrix between the two point clouds is obtained based on the rotation matrix and the center of mass translation matrix. At the same time, the average distance between the candidate point cloud data and the second point cloud data can be calculated, and the average distance can be used to represent the similarity between the candidate point cloud data and the second point cloud data. Therefore, through this process, the similarity and transformation matrix corresponding to each candidate point cloud data can be calculated separately.

[0545] There are many ways to calculate the average distance between two point clouds, which are not limited in this specification. The following is a method for calculating the average distance between candidate point cloud data and second point cloud data based on a KD tree (K-dimensional tree).

[0546] Taking candidate point cloud data along a particular principal axis as an example, we first construct a KD tree for the candidate point cloud data. A KD tree is a binary tree with each node representing a point. Next, for each point in the second point cloud data, we use the KD tree to perform a nearest neighbor search to find the distance to the closest point in the candidate point cloud data. Next, we sum the nearest neighbor distances of all query points and divide this by the total number of query points to obtain the average distance between the candidate point cloud data and the second point cloud data.

[0547] The above method can be used to calculate the average distance between each candidate point cloud data and the second point cloud data. The average distance represents the similarity between the candidate point cloud data and the second point cloud data. A smaller average distance indicates a higher similarity, that is, the closer the principal axis direction of the candidate point cloud data is to the second point cloud data. Conversely, a larger average distance indicates a lower similarity, that is, the greater the difference between the principal axis direction of the candidate point cloud data and the second point cloud data.

[0548] S1540: Determine the candidate point cloud data with the greatest similarity as the target candidate point cloud data, and determine the transformation matrix corresponding to the target candidate point cloud data as the first transformation matrix.

[0549] As can be seen from the foregoing, after respectively calculating the similarity and transformation matrix corresponding to each candidate point cloud data, the candidate point cloud data with the greatest similarity can be determined as the target candidate point cloud data, and its corresponding transformation matrix is ​​the first transformation matrix.

[0550] In the embodiments of this specification, the first transformation matrix represents the spatial transformation relationship between the first digital model and the second digital model. In some embodiments, the first transformation matrix is ​​represented as PCA-Trans, and the first transformation matrix PCA-Trans represents the mapping relationship between points on the first point cloud data and the second point cloud data.

[0551] From the above, it can be seen that in the embodiment of this specification, by utilizing the PCA point cloud registration method combined with the main axis direction flipping, the impact of the main axis direction difference between the first digital model and the second digital model can be effectively alleviated, and the model registration accuracy can be improved.

[0552] In some embodiments, after obtaining the first transformation matrix PCA-Trans through the aforementioned method process, the first transformation matrix PCA-Trans can be directly used to perform spatial position transformation on the first point cloud data to obtain a target digital model.

[0553] In other embodiments, in order to further improve the registration accuracy, the above-mentioned PCA algorithm based on the principal axis direction adjustment can be first used as a rough registration for the first point cloud data, and then further combined with the SICP algorithm as a fine registration of the point cloud to obtain a more accurate point cloud registration result, which is explained below in conjunction with Figure 17.

[0554] As shown in FIG17 , in some embodiments, the spatial transformation relationship between the first digital model and the second digital model includes not only the first transformation matrix described above, but also the second transformation matrix. The process of performing model registration on the first digital model and the second digital model includes:

[0555] S1710 , performing point cloud registration based on sparse iterative nearest points on the target candidate point cloud data and the second point cloud data to obtain a second transformation matrix.

[0556] The SICP algorithm, or sparse iterative closest point algorithm, is primarily used to address the registration problem of large-scale point cloud data. It iteratively finds the best match between the source and target point clouds to achieve point cloud alignment. In the disclosed embodiments, the SICP algorithm can be used to further align target candidate point cloud data that has undergone principal axis adjustment and PCA registration.

[0557] Specifically, in some embodiments, the adjustment of SICP registration can be set as follows: if the number of registrations is less than k1, the minimum distance d1 between each point in the target candidate point cloud data and the corresponding point in the target point cloud data is calculated. If d1>k2*L1, the point is considered to be an outlier and deleted, completing an update of the target candidate point cloud data until the number of registrations is greater than or equal to k1, completing the SICP registration process, and obtaining the second transformation matrix, which can be expressed as SICP-Trans.

[0558] In one example, parameter k1 is set to 3, k2 is initially set to 6, and the value of k2 is decremented by 2 after each iteration. L1 is the grid side length in the embodiment of FIG13 . Furthermore, variable parameters for SICP registration include the norm p and the number of iterations n. It is understood that the values ​​of these parameters can be adjusted based on specific scenario requirements and are not limited to the examples described above.

[0559] In the implementation manner of this specification, through the above-mentioned SICP point cloud registration process, the target candidate point cloud data and the second point cloud data can be further finely registered to obtain a second transformation matrix SICP-Trans between the two.

[0560] After obtaining the first transformation matrix PCA-Trans and the second transformation matrix SICP-Trans through the above method, the occlusal relationship of the original model can be adjusted based on the first transformation matrix PCA-Trans and the second transformation matrix SICP-Trans.

[0561] S1720: Perform spatial position transformation on the first point cloud data according to the first transformation matrix to obtain third point cloud data.

[0562] S1730. Perform spatial position transformation on the third point cloud data according to the second transformation matrix to obtain a target digital model.

[0563] In some embodiments of this specification, the spatial transformation relationship between the first digital model and the second digital model includes the aforementioned first transformation matrix PCA-Trans and second transformation matrix SICP-Trans. As can be seen from the foregoing, the first transformation matrix PCA-Trans is the transformation matrix for coarse registration of the first digital model, and the second transformation matrix SICP-Trans is the transformation matrix for fine registration of the first digital model.

[0564] Therefore, in the embodiments of this specification, when performing occlusal adjustment on the first digital model, the points on the first point cloud data of the first digital model can first be spatially transformed based on the first transformation matrix PCA-Trans to achieve coarse registration of the first digital model and obtain third point cloud data. Then, based on this, the points on the third point cloud data can be spatially transformed based on the second transformation matrix SICP-Trans to achieve fine registration of the third point cloud data, resulting in the final occlusally adjusted target digital model. The target digital model is an oral model that includes an occlusal relationship that is identical or close to the user's actual occlusal relationship.

[0565] As can be seen from the foregoing, in the embodiments of this specification, by adjusting the principal axis orientation of the first digital model, the orientation of the first digital model and the second digital model is kept as consistent or close as possible without manual intervention. This not only accelerates the model registration process, allowing for rapid convergence of the calculation results, but also improves the accuracy of the model registration, thereby improving the accuracy of the occlusal adjustment results. Furthermore, in the embodiments of this specification, a model registration method combining PCA coarse registration based on principal axis orientation adjustment with SICP fine registration is employed to improve the accuracy of the model registration results, thereby improving the accuracy of the occlusal adjustment results.

[0566] In one example, referring to FIG18 , FIG18 shows a visualization result of the model registration of the first digital model and the second digital model. It can be seen from FIG18 that the model registration in the embodiment of the present specification has good accuracy.

[0567] In one example, Figure 19 shows a comparison of the effects of the occlusion adjustment method described herein. Figure 19 (a) shows the occlusion visualization of the original model before adjustment using the method described herein, while Figure 19 (b) shows the occlusion visualization of the model after adjustment using the method described herein. Comparing Figures 19 (a) and (b) shows that the model after adjustment using the method described herein has a more accurate occlusion, resulting in an adjustment result that is closer to the user's actual occlusion.

[0568] In some embodiments, this specification provides an occlusal adjustment device that can be implemented in an electronic device. In the embodiments of this specification, the electronic device can be any suitable device type, such as an intraoral scanner, a computer, a server, a mobile terminal, a wearable device, etc., and this specification does not limit this.

[0569] As shown in FIG20 , in some embodiments, the occlusal adjustment device exemplified in this specification includes:

[0570] The model acquisition module 10 is configured to acquire a first digital model and a second digital model of the user's oral cavity, wherein the first digital model represents a digital model obtained by digitizing the user's oral cavity, and the second digital model represents a digital model obtained by collecting oral data of the user's oral cavity in an occluded state;

[0571] A model registration module 20 is configured to perform model registration on the first digital model and the second digital model to obtain a spatial transformation relationship for mapping the first digital model to the second digital model;

[0572] The occlusion adjustment module 30 is configured to adjust the occlusion relationship of the first digital model based on the spatial transformation relationship to obtain a target digital model of the user's oral cavity.

[0573] In some embodiments, the model acquisition module 10 is configured to:

[0574] Digitally processing the user's oral cavity to obtain an oral cavity model, and segmenting the tooth region of the oral cavity model to obtain the first digital model;

[0575] The user's oral cavity in the occlusal state is digitally processed to obtain an occlusal model, and the tooth area of ​​the occlusal model is segmented to obtain the second digital model.

[0576] In some embodiments, the model registration module 20 is configured to:

[0577] Direction matching is performed on the main axis directions of the first digital model and the second digital model, and model registration is performed based on the first digital model and the second digital model after direction matching to obtain the spatial conversion relationship.

[0578] In some embodiments, the model registration module 20 is configured to:

[0579] Acquire first point cloud data corresponding to the first digital model, and second point cloud data corresponding to the second digital model;

[0580] spatially flipping the principal axis of the first point cloud data to obtain candidate point cloud data in multiple principal axis directions;

[0581] Performing point cloud registration based on principal component analysis on each candidate point cloud data and the second point cloud data, respectively, to obtain a similarity and a transformation matrix corresponding to each candidate point cloud data, wherein the similarity represents the similarity between the candidate point cloud data and the second point cloud data;

[0582] The candidate point cloud data with the greatest similarity is determined as the target candidate point cloud data, and the transformation matrix corresponding to the target candidate point cloud data is determined as the first transformation matrix.

[0583] In some embodiments, the model registration module 20 is configured to:

[0584] Processing the first point cloud data to obtain a first eigenvector of the centroid of the first point cloud data, where a direction of the first eigenvector represents a principal axis direction of the first point cloud data;

[0585] The first point cloud data is flipped based on the main axis direction represented by the first eigenvector to obtain candidate point cloud data in multiple main axis directions.

[0586] In some embodiments, the model registration module 20 is configured to:

[0587] Performing point cloud registration on each candidate point cloud data and the second point cloud data respectively, and calculating an average distance between each candidate point cloud data and the second point cloud data, wherein the similarity includes the average distance;

[0588] The step of determining the candidate point cloud data with the greatest similarity as the target candidate point cloud data includes:

[0589] The candidate point cloud data with the smallest average distance is determined as the target candidate point cloud data.

[0590] In some embodiments, the bite adjustment module 30 is configured to:

[0591] The first point cloud data is spatially transformed according to the first transformation matrix to obtain the target digital model.

[0592] In some embodiments, the model registration module 20 is configured to:

[0593] The target candidate point cloud data and the second point cloud data are subjected to point cloud registration based on sparse iterative nearest point to obtain the second transformation matrix.

[0594] In some embodiments, the bite adjustment module 30 is configured to:

[0595] Performing spatial position transformation on the first point cloud data according to the first transformation matrix to obtain third point cloud data;

[0596] The third point cloud data is spatially transformed according to the second transformation matrix to obtain the target digital model.

[0597] In some embodiments, the first digital model and the second digital model include any one of an upper jaw model, a lower jaw model, an upper and lower jaw model, and a tooth model of the user's mouth.

[0598] Specifically, the first digital model and the second digital model should be the same type of models, that is, the first digital model and the second digital model are both maxillary models; or the first digital model and the second digital model are both mandibular models.

[0599] In some embodiments, this specification provides an electronic device, including:

[0600] processor; and

[0601] A memory stores computer instructions, wherein the computer instructions are used to enable the processor to execute the method described in any of the above embodiments.

[0602] In some embodiments, this specification provides a storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the method described in any of the above embodiments.

[0603] In some embodiments, this specification provides a computer program product, which implements the method described in any of the above embodiments when running.

[0604] Specifically, FIG21 shows a schematic structural diagram of an electronic device 2100 suitable for implementing the method of this specification. The electronic device shown in FIG21 can realize the corresponding functions of the above-mentioned processor and storage medium.

[0605] As shown in Figure 21, electronic device 2100 includes a processor 2101, which can perform various appropriate actions and processes according to the program stored in memory 2102 or the program loaded from storage portion 2108 into memory 2102. Various programs and data required for the operation of electronic device 2100 are also stored in memory 2102. Processor 2101 and memory 2102 are connected to each other via bus 2104. Input / output (I / O) interface 2105 is also connected to bus 2104.

[0606] The following components are connected to the I / O interface 2105: an input section 2106 including a keyboard, a mouse, and the like; an output section 2107 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 2108 including a hard disk; and a communication section 2109 including a network interface card such as a LAN card or a modem. The communication section 2109 performs communication processing via a network such as the Internet. A drive 2110 is also connected to the I / O interface 2105 as needed. Removable media 2111, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 2110 as needed, so that computer programs read therefrom can be installed into the storage section 2108 as needed.

[0607] In particular, according to embodiments of the present specification, the above method process can be implemented as a computer software program. For example, embodiments of the present specification include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program comprising program code for executing the above method. In such embodiments, the computer program can be downloaded and installed from a network via the communication portion 2109 and / or installed from removable media 2111.

[0608] It should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each implementation method can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

[0609] Obviously, the above embodiments are merely examples for clarity of explanation and are not intended to limit the embodiments. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all embodiments here. Obvious variations or modifications arising therefrom remain within the scope of protection created by this specification.

Claims

1. A method for dental arch recognition, characterized in that, Including: Obtaining first dental arch model data and second dental arch model data of a tooth to be measured respectively, where the first dental arch model and the second dental arch model correspond to different dental arches; Obtaining relationship identification information of tooth pairs of the tooth to be measured according to the first dental arch model data and the second dental arch model data, where the tooth pairs include at least one pair of teeth located in different dental arches and corresponding to each other, and the relationship identification information is used to indicate the relative position relationship of the tooth pairs; Determining the dental arch information corresponding to the first dental arch model data and the second dental arch model data according to the relationship identification information.

2. The dental arch recognition method according to claim 1, wherein, The dental arch information includes maxilla determination information or mandible determination information for the first dental arch model, and maxilla determination information or mandible determination information for the second dental arch model.

3. The dental arch recognition method according to claim 1 or 2, characterized in that The obtaining relationship identification information of tooth pairs of the tooth to be measured according to the first dental arch model data and the second dental arch model data includes: Obtaining tooth pair data of at least one pair of teeth located in different dental arches and corresponding to each other based on the first dental arch model data and the second dental arch model data; Parallelly inputting the tooth pair data into a dental arch relationship recognition model to predict corresponding relationship identification information; the dental arch relationship recognition model is a neural network model, and the dental arch relationship recognition model is trained based on preset tooth pair data and relationship labels representing the relative position relationship of tooth pairs in the preset tooth pair data.

4. The method for dental arch recognition according to claim 3, characterized in that, The parallelly inputting the tooth pair data into a dental arch relationship recognition model to predict corresponding relationship identification information includes: Extracting features from the tooth data to be measured according to a first feature extraction module to obtain first tooth data, and extracting features from the tooth data to be measured according to a second feature extraction module to obtain second tooth data; Obtaining first feature data and second feature data corresponding to the first tooth data and the second tooth data respectively, and generating a tooth pair feature sequence; Predicting relationship identification information of the first tooth and the second tooth according to the tooth pair feature sequence.

5. The method for dental arch recognition according to any one of claims 1-4, characterized in that The determining the dental arch information corresponding to the first dental arch model data and the second dental arch model data according to the relationship identification information includes: Obtaining a numerical form of the relationship identification information, and determining the maxilla and mandible determination information of the first dental arch model data and the second dental arch model data according to the numerical relationship.

6. The method for dental arch recognition according to claim 5, wherein, The numerical form of the relationship identification information includes: When the tooth to be measured in the first dental arch model points to an upper jaw tooth and the tooth to be measured in the second dental arch model data points to a lower jaw tooth in the tooth pair data, obtaining relationship identification information with a numerical form of a first label value; When the tooth to be measured in the first dental arch model points to a lower jaw tooth and the tooth to be measured in the second dental arch model data points to an upper jaw tooth in the tooth pair data, obtaining relationship recognition information with a numerical form of a second label value.

7. The dental arch recognition method according to claim 5, wherein The determining the maxilla and mandible determination information of the first dental arch model data and the second dental arch model data according to the numerical relationship includes: Based on the label values in the numerical form corresponding to multiple groups of tooth pairs, determining the maxilla and mandible determination information according to the voting scoring result.

8. The dental arch recognition method according to any one of claims 1-7, characterized in that, Any tooth pair data is obtained through the following method, including: Segment the first dental arch model and the second dental arch model to obtain a number of first dental arch tooth data and a number of second dental arch tooth data; Match the first dental arch tooth data and the second dental arch tooth data according to the tooth position number to obtain tooth pair data.

9. The method for dental arch recognition according to claim 8, wherein, The segmenting the first dental arch model and the second dental arch model includes: Perform surface simplification on the first dental arch model and the second dental arch model according to the differentiability of parts in the dental arch model; Register the first dental arch model and the second dental arch model respectively, and segment the first dental arch model and the second dental arch model based on the tooth segmentation model, where the tooth segmentation model is a neural network model trained based on a preset dental arch model and a preset tooth label representing the tooth distribution in the preset dental arch model.

10. The dental arch recognition method according to claim 9, characterized in that, The performing surface simplification on the first dental arch model and the second dental arch model according to the differentiability of parts in the dental arch model includes: Determine the patch segmentation method of the dental arch model according to the differentiability of adjacent patches in the dental arch model, and obtain the patches of the dental arch model accordingly; Merge the effective vertices of the obtained patches.

11. The method for dental arch recognition according to claim 10, wherein The determining the patch segmentation method of the dental arch model according to the differentiability of adjacent patches in the dental arch model includes: Determine the patch segmentation method of the dental arch model according to the differentiability between the patch and other intraoral tissues and the cost of adjacent patches belonging to the same intraoral tissue; the differentiability between the patch and other intraoral tissues is determined according to at least one of the following indicators: The degree to which the current patch is close to the incisal edge connection line; The degree to which the current patch is close to the incisal extreme point of the corresponding tooth; The degree to which the current patch is far from the model center of the dental arch model data.

12. A display method, characterized in that, Includes: Display the first dental arch model and the second dental arch model according to the dental arch information, where the dental arch information is obtained based on the dental arch recognition method according to any one of claims 1-11; The first dental arch model and the second dental arch model are used to design an orthodontic treatment plan.

13. An oral instrument, characterized in that, The oral appliance is prepared based on the dental arch information determined by the dental arch recognition method according to any one of claims 1-11.

14. A computer storage medium, on which an application program is stored, characterized in that, When the application program is executed, the steps of the dental arch recognition method according to any one of claims 1-11 are implemented.

15. An occlusal recognition device, characterized in that, Includes: A first module for obtaining first dental arch model data and second dental arch model data of a to-be-detected tooth; The first dental arch model and the second dental arch model correspond to different dental arches; A second module for obtaining relationship identification information of a tooth pair of the to-be-detected tooth according to the first dental arch model data and the second dental arch model data, where the tooth pair includes at least one pair of teeth located in different dental arches and corresponding to each other, and the relationship identification information is used to indicate the relative position relationship of the tooth pair; A third module for determining the dental arch information corresponding to the first dental arch model data and the second dental arch model data according to the relationship identification information.

16. An electronic device, characterized in that, Includes a processor, a memory and a communication bus, characterized in that the processor and the memory complete communication with each other through the communication bus; The memory is used for storing an application program; The processor is used for implementing the steps of the dental arch recognition method according to any one of claims 1-11 when executing the application program stored on the memory.

17. A method for bite adjustment, characterized in that, Includes: Obtain a first digital model and a second digital model of the user's oral cavity, where the first digital model represents the digital model obtained by digitizing the user's oral cavity, and the second digital model represents the digital model obtained after collecting oral cavity data in the occlusal state of the user's oral cavity; Perform model registration on the first digital model and the second digital model to obtain a spatial transformation relationship for mapping the first digital model to the second digital model; Adjust the occlusal relationship of the first digital model based on the spatial transformation relationship to obtain the target digital model of the user's oral cavity.

18. The method according to claim 17, wherein The first digital model and the second digital model are digital models of the tooth region of the user's oral cavity, and obtaining the first digital model and the second digital model of the user's oral cavity includes: Digitize the user's oral cavity to obtain an oral cavity model, and perform model segmentation on the tooth region of the oral cavity model to obtain the first digital model; Digitize the user's oral cavity in the occlusal state to obtain an occlusal model, and perform model segmentation on the tooth region of the occlusal model to obtain the second digital model.

19. The method according to claim 17, wherein The performing model registration on the first digital model and the second digital model to obtain a spatial transformation relationship for mapping the first digital model to the second digital model includes: Perform direction matching on the principal axis directions of the first digital model and the second digital model, and perform model registration on the first digital model and the second digital model after direction matching to obtain the spatial transformation relationship.

20. The method according to claim 19, wherein The spatial transformation relationship includes a first transformation matrix; the performing direction matching on the principal axis directions of the first digital model and the second digital model, and performing model registration on the first digital model and the second digital model after direction matching to obtain the spatial transformation relationship includes: Obtain the first point cloud data corresponding to the first digital model and the second point cloud data corresponding to the second digital model; Perform spatial flipping on the principal axis of the first point cloud data to obtain candidate point cloud data in multiple principal axis directions; Perform point cloud registration based on principal component analysis on each candidate point cloud data and the second point cloud data respectively to obtain the similarity and transformation matrix corresponding to each candidate point cloud data, where the similarity represents the similarity between the candidate point cloud data and the second point cloud data; Determine the candidate point cloud data with the maximum similarity as the target candidate point cloud data, and determine the transformation matrix corresponding to the target candidate point cloud data as the first transformation matrix.

21. The method according to claim 20, wherein The performing spatial flipping on the principal axis of the first point cloud data to obtain candidate point cloud data in multiple principal axis directions includes: Process the first point cloud data to obtain the first eigenvector of the centroid of the first point cloud data, and the direction of the first eigenvector represents the principal axis direction of the first point cloud data; Flip the first point cloud data based on the principal axis direction represented by the first eigenvector to obtain candidate point cloud data in multiple principal axis directions.

22. The method according to claim 20, wherein Performing point cloud registration based on principal component analysis on each candidate point cloud data and the second point cloud data respectively to obtain the similarity corresponding to each candidate point cloud data includes: Performing point cloud registration on each candidate point cloud data and the second point cloud data respectively, and calculating the average distance between each candidate point cloud data and the second point cloud data, where the similarity includes the average distance; Determining the candidate point cloud data with the maximum similarity as the target candidate point cloud data includes: Determining the candidate point cloud data with the minimum average distance as the target candidate point cloud data.

23. The method according to any one of claims 20 to 22, characterized in that, Adjusting the occlusion state of the first digital model based on the spatial transformation relationship to obtain the target digital model of the user's oral cavity includes: Performing spatial position transformation on the first point cloud data according to the first transformation matrix to obtain the target digital model.

24. The method according to any one of claims 20 to 22, characterized in that, The method further includes: Performing point cloud registration based on sparse iterative closest point on the target candidate point cloud data and the second point cloud data to obtain the second transformation matrix; Adjusting the occlusion state of the first digital model based on the spatial transformation relationship to obtain the target digital model of the user's oral cavity includes: Performing spatial position transformation on the first point cloud data according to the first transformation matrix to obtain the third point cloud data; Performing spatial position transformation on the third point cloud data according to the second transformation matrix to obtain the target digital model.

25. The method according to claim 17, wherein The first digital model and the second digital model are any one of the maxillary model, mandibular model, maxillomandibular model, and tooth model of the user's oral cavity.

26. An occlusal adjustment device, characterized in that, Includes: A model acquisition module configured to acquire a first digital model and a second digital model of the user's oral cavity, where the first digital model represents a digital model obtained by digitizing the user's oral cavity, and the second digital model represents a digital model obtained after collecting oral cavity data in the occlusal state of the user's oral cavity; A model registration module configured to perform model registration on the first digital model and the second digital model to obtain a spatial transformation relationship for mapping the first digital model to the second digital model; An occlusion adjustment module configured to adjust the occlusion relationship of the first digital model based on the spatial transformation relationship to obtain the target digital model of the user's oral cavity.

27. An electronic device, characterized in that, Includes: A processor; And A memory storing computer instructions for causing the processor to execute the method according to any one of claims 17 to 25.

28. A storage medium, characterized in that, Storing computer instructions for causing a computer to execute the method according to any one of claims 17 to 25.

29. A computer program product, characterized in that, The computer program product implements the method according to any one of claims 17 to 25 when running.

Images