Oral cavity 3D modeling printing method based on multi-modal neural network

Abstract

According to the oral cavity 3D modeling printing method based on the multi-modal neural network, firstly, target CT image data are obtained, oral cavity tooth model data are obtained based on the target CT image data, then target intraoral data are obtained, the target intraoral data comprise occlusal surface data, buccal side data and lingual side data, and then the target intraoral data are obtained. And obtaining a trained neural network model, importing the oral tooth model data, the target intraoral data and the target facial data into the neural network model, obtaining an output result of the neural network model, and finally determining surgical 3D modeling printing parameters based on the output result. According to the oral 3D modeling and printing method based on the multi-modal neural network provided by the embodiment of the invention, the operation 3D modeling and printing parameters are determined through the neural network model in combination with multiple data, manual operation steps are remarkably reduced, and the whole process planning from data processing to scheme generation is automatically completed through the deep learning model.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to an oral 3D modeling and printing method based on a multimodal neural network. Background Art

[0002] Oral implant restoration is one of the mainstream treatment methods for restoring missing teeth today. Implant placement is a complex treatment process that requires consideration of multiple key factors, including anatomical structure (such as the mandibular nerve canal and maxillary sinus), bone tissue conditions (such as bone mass and bone quality), soft tissue conditions (such as gum thickness and morphology), and the final position of the restoration. Clinical practice has shown that improper implant position not only affects the restoration effect, but may also lead to surgical complications, which is one of the important reasons for the failure of implant treatment.

[0003] Before performing oral implant restoration, the surgery can be simulated digitally in advance, that is, a 3D model of the surgical process can be generated by digital 3D modeling, and the model can be used for surgical rehearsal or printed as a physical model for discussion. For the establishment of digital models, related technologies only fill in the data in a single way, and the surgical process cannot be intuitively reflected on the model. Summary of the invention

[0004] The embodiment of the present application provides an oral 3D modeling and printing method based on a multimodal neural network to improve the above-mentioned problems.

[0005] In order to achieve the above objectives, this application adopts the following technical solutions: In a first aspect, an embodiment of the present application provides an oral 3D modeling and printing method based on a multimodal neural network, comprising: Acquire target CT image data, and acquire oral tooth model data based on the target CT image data, wherein the target CT image data is CBCT voxel data; Acquire target intra-oral data, which includes occlusal surface data, buccal data, and lingual data, wherein the target intra-oral data is oral scan point cloud data; Acquire target facial data, where the target facial data is facial soft tissue data; Obtaining a trained neural network model, importing oral tooth model data, target intraoral data, and target facial data into the neural network model, and obtaining an output result of the neural network model; Based on the output results, the surgical 3D modeling printing parameters are determined.

[0006] In conjunction with the first aspect, in some embodiments, the method further includes: Obtain preset surgical plans, including: single tooth implant, multiple tooth implant, maxillary sinus floor lift and immediate implant; Based on the output results, determine the surgical 3D modeling printing parameters, including: When the preset surgical plan is a single missing tooth implant, the surgical 3D modeling and printing parameters are the single missing tooth implant angle; When the preset surgical plan is to implant multiple missing teeth, the surgical 3D modeling and printing parameters are the implantation angles of the multiple missing teeth and the arrangement order of the multiple missing teeth; When the preset surgical plan is maxillary sinus floor lift and immediate implantation, the surgical 3D modeling and printing parameters are safety thresholds, which include the cutting depth.

[0007] In conjunction with the first aspect, in some embodiments, the method further includes: Before obtaining the trained neural network model, importing the oral tooth model data, the target intraoral data and the target facial data into the neural network model, and obtaining the output result of the neural network model, the following steps are also included: A training set is obtained, and an initial neural network model is trained based on the training set. When the initial neural network model meets preset conditions, the training is stopped, and the initial neural network is used as the neural network model.

[0008] In conjunction with the first aspect, in some embodiments, the method further includes: Acquire target CT image data, and acquire oral tooth model data based on the target CT image data, wherein the target CT image data is CBCT voxel data, including: Acquire a plurality of adjacent oral CT images, and determine an oral digital model based on the plurality of oral CT images; Determine the target area based on the oral digital model, wherein the target area is a partial model in the oral digital model, and the target area includes all teeth and gums; determining a first CT image, wherein a scanning surface for acquiring the first CT image interferes with the target area; Acquiring a second CT image, wherein a scanning plane for acquiring the second CT image and a scanning plane for acquiring the first CT image are adjacent scanning planes; Obtaining an average grayscale value Qv1 of the first CT image and an average grayscale value Qv2 of the second CT image; Segmenting the first CT image to form a plurality of first target images, and obtaining a gray value Q1 of each first target image, segmenting the second CT image to form a plurality of second target images, and obtaining a gray value Q2 of each second target image; Obtaining a grayscale difference H1 between a plurality of average grayscale values ​​Qv1 and grayscale value Q1, and obtaining a grayscale difference H2 between a plurality of average grayscale values ​​Qv2 and grayscale value Q2; Overlap the first target image and the second target image at the same position, and determine the tooth boundary according to the grayscale difference H1 and the grayscale difference H2; Determine the single tooth model based on the tooth boundaries.

[0009] In conjunction with the first aspect, in some embodiments, the method further includes: Overlapping the first target image and the second target image at the same position, and determining the tooth boundary according to the grayscale difference H1 and the grayscale difference H2, including: Acquire the first target image and the second target image at the overlapping position, and determine the variable Ht between the grayscale difference H1 and the grayscale difference H2; Whether the overlapping position is a tooth boundary is determined based on the variable Ht.

[0010] In conjunction with the first aspect, in some embodiments, the method further includes: Determining whether the overlapping position is a tooth boundary based on the variable Ht includes: Get the average variable h between the average gray value Qv1 and the average gray value Qv2; The variable Ht is compared with the average variable h. If the variable Ht is greater than the average variable h, it is determined that the tooth boundary is located between the scanning surface for acquiring the first target image and the scanning surface for acquiring the second target image.

[0011] In conjunction with the first aspect, in some embodiments, the method further includes: The target area is determined based on the oral digital model, wherein the target area is a partial model in the oral digital model, and the target area includes all teeth and gums, including: Decompose the oral digital model into multiple sub-models; Determine the degree of adhesion between multiple sub-models based on fuzzy algorithm; Determining components of the oral digital model based on the adhesion degree, wherein sub-models with the same adhesion degree are one component; One or more components among the multiple components are determined as target areas through boundary recognition.

[0012] In conjunction with the first aspect, in some embodiments, the method further includes: One or more components among the multiple components are determined as target areas through boundary recognition, including: Obtaining a preset model set, the preset model set including tooth model data and gum model data; Match the component with the preset model set and obtain the matching result; According to the matching results, the target area is determined.

[0013] In a second aspect, an embodiment of the present application provides an oral 3D modeling and printing system based on a multimodal neural network, the system being configured as follows: Acquire target CT image data, and acquire oral tooth model data based on the target CT image data, wherein the target CT image data is CBCT voxel data; Acquire target intra-oral data, which includes occlusal surface data, buccal data, and lingual data, wherein the target intra-oral data is oral scan point cloud data; Acquire target facial data, where the target facial data is facial soft tissue data; Obtaining a trained neural network model, importing oral tooth model data, target intraoral data, and target facial data into the neural network model, and obtaining an output result of the neural network model; Based on the output results, the surgical 3D modeling printing parameters are determined.

[0014] In conjunction with the second aspect, in some embodiments, the system is configured as follows: Obtain preset surgical plans, including: single tooth implant, multiple tooth implant, maxillary sinus floor lift and immediate implant; Based on the output results, determine the surgical 3D modeling printing parameters, including: When the preset surgical plan is a single missing tooth implant, the surgical 3D modeling and printing parameters are the single missing tooth implant angle; When the preset surgical plan is to implant multiple missing teeth, the surgical 3D modeling and printing parameters are the implantation angles of the multiple missing teeth and the arrangement order of the multiple missing teeth; When the preset surgical plan is maxillary sinus floor lift and immediate implantation, the surgical 3D modeling and printing parameters are safety thresholds, which include the cutting depth.

[0015] In conjunction with the second aspect, in some embodiments, the system is configured as follows: Before obtaining the trained neural network model, importing the oral tooth model data, the target intraoral data and the target facial data into the neural network model, and obtaining the output result of the neural network model, the following steps are also included: A training set is obtained, and an initial neural network model is trained based on the training set. When the initial neural network model meets preset conditions, the training is stopped, and the initial neural network is used as the neural network model.

[0016] In conjunction with the second aspect, in some embodiments, the system is configured as follows: Acquire target CT image data, and acquire oral tooth model data based on the target CT image data, wherein the target CT image data is CBCT voxel data, including: Acquire a plurality of adjacent oral CT images, and determine an oral digital model based on the plurality of oral CT images; Determine the target area based on the oral digital model, wherein the target area is a partial model in the oral digital model, and the target area includes all teeth and gums; determining a first CT image, wherein a scanning surface for acquiring the first CT image interferes with the target area; Acquiring a second CT image, wherein a scanning plane for acquiring the second CT image and a scanning plane for acquiring the first CT image are adjacent scanning planes; Obtaining an average grayscale value Qv1 of the first CT image and an average grayscale value Qv2 of the second CT image; Segmenting the first CT image to form a plurality of first target images, and obtaining a gray value Q1 of each first target image, segmenting the second CT image to form a plurality of second target images, and obtaining a gray value Q2 of each second target image; Obtaining a grayscale difference H1 between a plurality of average grayscale values ​​Qv1 and grayscale value Q1, and obtaining a grayscale difference H2 between a plurality of average grayscale values ​​Qv2 and grayscale value Q2; Overlap the first target image and the second target image at the same position, and determine the tooth boundary according to the grayscale difference H1 and the grayscale difference H2; Determine the single tooth model based on the tooth boundaries.

[0017] In conjunction with the second aspect, in some embodiments, the system is configured as follows: Overlapping the first target image and the second target image at the same position, and determining the tooth boundary according to the grayscale difference H1 and the grayscale difference H2, including: Acquire the first target image and the second target image at the overlapping position, and determine the variable Ht between the grayscale difference H1 and the grayscale difference H2; Whether the overlapping position is a tooth boundary is determined based on the variable Ht.

[0018] In conjunction with the second aspect, in some embodiments, the system is configured as follows: Determining whether the overlapping position is a tooth boundary based on the variable Ht includes: Get the average variable h between the average gray value Qv1 and the average gray value Qv2; The variable Ht is compared with the average variable h. If the variable Ht is greater than the average variable h, it is determined that the tooth boundary is located between the scanning surface for acquiring the first target image and the scanning surface for acquiring the second target image.

[0019] In conjunction with the second aspect, in some embodiments, the system is configured as follows: The target area is determined based on the oral digital model, wherein the target area is a partial model in the oral digital model, and the target area includes all teeth and gums, including: Decompose the oral digital model into multiple sub-models; Determine the degree of adhesion between multiple sub-models based on fuzzy algorithm; Determining components of the oral digital model based on the adhesion degree, wherein sub-models with the same adhesion degree are one component; One or more components among the multiple components are determined as target areas through boundary recognition.

[0020] In conjunction with the second aspect, in some embodiments, the system is configured as follows: One or more components among the multiple components are determined as target areas through boundary recognition, including: Obtaining a preset model set, the preset model set including tooth model data and gum model data; Match the component with the preset model set and obtain the matching result; According to the matching results, the target area is determined.

[0021] A third aspect of an embodiment of the present invention provides an electronic device, the electronic device comprising: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method proposed in the first aspect of the embodiment of the present invention.

[0022] A fourth aspect of an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method provided in the first aspect of the embodiment of the present invention.

[0023] In summary, the above method and device have the following technical effects: The embodiment of the present application proposes an oral 3D modeling and printing method based on a multimodal neural network. First, the target CT image data is obtained, and the oral tooth model data is obtained based on the target CT image data, wherein the target CT image data is CBCT voxel data, and then the target intraoral data is obtained, and the target intraoral data includes occlusal surface data, buccal data and lingual data, wherein the target intraoral data is oral scan point cloud data, and then the target facial data is obtained, and the target facial data is facial soft tissue data, and then the trained neural network model is obtained, and the oral tooth model data, the target intraoral data and the target facial data are imported into the neural network model, and the output result of the neural network model is obtained, and finally, the surgical 3D modeling and printing parameters are determined based on the output result. The embodiment of the present application proposes an oral 3D modeling and printing method based on a multimodal neural network, which determines the surgical 3D modeling and printing parameters by combining multiple data through a neural network model, significantly reduces the manual operation steps, and automatically completes the full process planning from data processing to solution generation through a deep learning model. The system can automatically identify key anatomical structures, calculate optimal implant parameters and generate execution files, greatly reducing the reliance on doctors' experience, significantly improving planning efficiency, and enabling doctors to devote more energy to program evaluation and optimization. Through multimodal data fusion technology, accurate registration and deep integration of CBCT, oral scan and facial scan data are achieved. The system uses advanced deep learning algorithms for feature extraction and analysis, which significantly improves the accuracy of anatomical structure recognition and effectively avoids the errors that may be caused by traditional manual measurement, thereby ensuring the reliability and accuracy of the planning plan. Taking full account of individual differences in patients, it can automatically generate personalized implant plans based on the anatomical characteristics, bone conditions and aesthetic needs of specific cases. The system optimizes the implant position, angle and depth through multi-dimensional analysis, and combines facial soft tissue data for aesthetic evaluation, realizing truly patient-specific planning. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A schematic flow chart of an oral 3D modeling and printing method based on a multimodal neural network proposed in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] In recent years, artificial intelligence technology has made significant progress in the medical field. Deep learning algorithms have demonstrated superior performance in medical image analysis, lesion detection, and auxiliary diagnosis. Especially in the field of radiological imaging, AI technology has been able to assist doctors in lesion identification and three-dimensional reconstruction, improving diagnostic efficiency and accuracy. At the same time, by analyzing large-scale clinical data, AI systems can provide doctors with personalized treatment plan recommendations. However, there is currently no mature solution for applying deep learning technology, especially end-to-end convolutional neural networks, to oral implant surgery planning. Existing intelligent auxiliary systems are mostly limited to a single function, such as bone density analysis or anatomical structure recognition, and lack comprehensive support for the entire implant planning process.

[0027] This application embodiment proposes a method for oral 3D modeling and printing based on a multimodal neural network. Figure 1 , including the following steps: S101: Acquire target CT image data, and acquire oral tooth model data based on the target CT image data, wherein the target CT image data is CBCT voxel data.

[0028] Optionally, before this, it may also include: obtaining a preset surgical plan, which includes: single-tooth implantation, multiple-tooth implantation, maxillary sinus floor lifting, and immediate implantation.

[0029] It is understandable that a standardized CBCT scanning process can be used for CBCT voxel data, including: Control scanning parameters: tube voltage was set to 80-120 kV, tube current was set to 5-8 mA; Set voxel resolution: control it within the range of 0.15-0.3mm; Perform quality control: monitor motion artifacts, metal artifacts, image contrast and noise levels in real time; Make sure that the scan fully covers the jaw area to be evaluated.

[0030] As for how to obtain model data through images, as an implementation method, in this embodiment, the following method can be used: Acquire multiple adjacent oral CT images, and determine the oral digital model based on the multiple oral CT images. It is understandable that the specific structures in the oral cavity can be preliminarily modeled through multiple CT images, and the specific modeling steps have been disclosed in the relevant technology and will not be repeated here.

[0031] The target area is determined based on the oral digital model, wherein the target area is a partial model in the oral digital model, and the target area includes all teeth and gums. It is understandable that after obtaining the completed oral digital model, it is necessary to select the target area for more accurate modeling, and the target area is some specific parts in the oral model. In this embodiment, teeth and gums are taken as an example.

[0032] It can be understood that, as an implementation method, the oral digital model can be decomposed into multiple sub-models. The model can be divided into multiple overlapping sub-models by equal-quality segmentation. The number of sub-models is determined according to the specific processing capability of the processor. In this embodiment, the size of the sub-model is at least much smaller than the size of a single tooth.

[0033] Then, the degree of adhesion between multiple sub-models is determined based on the fuzzy algorithm. The oral digital model is composed of multiple structures of different sizes and shapes. Since different tissues have different absorption efficiencies of X-rays, there are rough boundaries between different tissues. The boundary line can be determined by the fuzzy algorithm. Therefore, after the oral digital model is decomposed into multiple sub-models, the degree of adhesion between sub-models belonging to the same tissue is higher.

[0034] Then, the components of the oral digital model are determined based on the adhesion degree. Based on the different adhesion degrees, the sub-models belonging to the same tissue can be marked by the adhesion degree, and then the sub-models with the same adhesion degree are combined to form a tissue of the same type composed of multiple sub-models. Since the fuzzy algorithm is used in this embodiment, the teeth and gums belong to the bone structure, and the rest belong to the soft tissue. By combining the sub-models, a three-dimensional area formed by the teeth and gums can be obtained, that is, the target area.

[0035] Among them, sub-models with the same adhesion degree are a component. In this process, a preset model set can be obtained, which includes tooth model data and gum model data. Then, the component is matched with the preset model set, and the matching result is obtained. Finally, the target area is determined according to the matching result. In the above manner, it is possible to determine one or more components of multiple components as the target area through boundary recognition. By recognizing the shape of the component, it is possible to identify which component is the three-dimensional area formed by the combination of teeth and gums.

[0036] A first CT image is determined, wherein a scanning surface for acquiring the first CT image interferes with a target area. Since the tooth model to be finally established belongs to the target area, the first CT image must interfere with the target area so that the first CT image has a partial cross-section of the target tooth.

[0037] A second CT image is acquired, wherein the scanning plane for acquiring the second CT image is an adjacent scanning plane to the scanning plane for acquiring the first CT image. When the model boundary crosses from one plane to another, the grayscale of the CT image will inevitably change dramatically. Based on the demand for model accuracy, when changes occur, the closer the second image acquisition plane is, the better, and the minimum distance between the two planes is determined according to the minimum spacing distance between the scanning planes during the CT image scanning process. Therefore, the scanning plane of the second CT image is an adjacent scanning plane to the scanning plane for acquiring the first CT image.

[0038] The average grayscale value Qv1 of the first CT image and the average grayscale value Qv2 of the second CT image are obtained, and the information displayed on the overall screen of the T image is the grayscale image formed by the device after scanning. By obtaining the average grayscale value Qv1 of the first CT image and the average grayscale value Qv2 of the second CT image, the overall grayscale situation can be determined, and when it changes, it can be used as a comparison parameter.

[0039] The first CT image is segmented to form a plurality of first target images, and a gray value Q1 of each first target image is obtained. The second CT image is segmented to form a plurality of second target images, and a gray value Q2 of each second target image is obtained. In this embodiment, the first target image and the second target image are segmented images. Based on the processing power of the computer, the smaller the first target image and the second target image are, the higher their accuracy is. In this embodiment, the first target image and the second target image are both taken as an example of one pixel.

[0040] Obtain the grayscale difference H1 between multiple average grayscale values ​​Qv1 and grayscale value Q1, obtain the grayscale difference H2 between multiple average grayscale values ​​Qv2 and grayscale value Q2, obtain the grayscale value of each pixel, and match it with its average grayscale value. When the grayscale value changes dramatically, it proves that a dramatic change has occurred in the image. Drastic changes only occur at the boundary where a single rigid body model appears or disappears. Therefore, in this embodiment, the grayscale difference H1 and the grayscale difference H2 of the pixels at the same position can be compared to determine whether it is a boundary position.

[0041] Overlap the first target image and the second target image at the same position, and determine the tooth boundary according to the grayscale difference H1 and the grayscale difference H2. The same position refers to the same point on each adjacent CT image during the same shooting process, that is, two adjacent CT images shot under the same field of view. If a two-dimensional coordinate system is established using the image, the same point represents the same coordinate.

[0042] Specifically, the first target image and the second target image at the overlapping position can be obtained to determine the variable Ht between the grayscale difference H1 and the grayscale difference H2. It can be understood that the average variable h between the average grayscale value Qv1 and the average grayscale value Qv2 is first obtained, and then the variable Ht is compared with the average variable h. If the variable Ht is greater than the average variable h, it is determined that the tooth boundary is located between the scanning surface for obtaining the first target image and the scanning surface for obtaining the second target image.

[0043] Then, whether the overlapping position is a tooth boundary can be determined based on the variable Ht.

[0044] Determine the single tooth model based on tooth boundaries; The tooth model data is obtained based on the tooth model. It can be understood that the tooth model data can be characteristic size data, surface data or graphic data constituting the tooth model, which is not limited here.

[0045] S102: Acquire target intra-oral data, which includes occlusal surface data, buccal data, and lingual data, wherein the target intra-oral data is oral scan point cloud data.

[0046] Specifically, a standard scanning sequence can be executed: collect occlusal, buccal, and lingual data in sequence; control the acquisition density: the point cloud data density is not less than 20 points / mm²; monitor data quality in real time: evaluate surface integrity and edge clarity; and use a multi-angle supplementary scanning strategy to ensure data integrity.

[0047] S103: Acquire target facial data, where the target facial data is facial soft tissue data.

[0048] Specifically, a multi-angle stereo acquisition solution is adopted, which includes: setting standard acquisition angles: including five positions of 0°, ±45°, and ±90°; arranging reference markers: for later data alignment; standardizing the acquisition environment: controlling lighting conditions to avoid interference and occlusion; and recording facial soft tissue data under standard expression states.

[0049] S104: Obtain a trained neural network model, import oral tooth model data, target intraoral data, and target facial data into the neural network model, and obtain the output result of the neural network model.

[0050] As can be understood, the network contains three parallel feature extraction branches, which process CBCT voxel data, oral scan point cloud data and facial soft tissue data respectively. The CBCT voxel feature extraction module is based on the improved 3D ResNet structure, and realizes accurate recognition of anatomical structure through multi-scale feature extraction and spatial attention mechanism; the oral scan point cloud feature extraction module adopts an improved point cloud processing network, combining local geometric features and global structural features to fully express the spatial relationship of oral anatomy; the facial soft tissue feature extraction module integrates multi-angle facial features and three-dimensional contour information through a 2D / 3D hybrid feature network to provide a basis for aesthetic design. The cross-modal attention module realizes the dynamic fusion of different modal features through an adaptive weight mechanism, while ensuring feature alignment at the spatial and semantic levels; the multi-level fusion strategy realizes the deep fusion of multimodal information through feature integration in the early, middle and late stages, combined with global context information.

[0051] The implant planning prediction network includes three branches: position prediction, angle prediction, and specification prediction. The position prediction branch uses the spatial positioning network and the anatomical constraint module to accurately predict the implant position; the angle prediction branch combines the direction prediction network and the biomechanical optimization module to determine the optimal implant angle; the specification prediction branch uses a multi-task learning framework to complete the intelligent selection of implant specifications under clinical constraints.

[0052] As for its training process, optionally, a training set can be obtained first, and the initial neural network model can be trained based on the training set. When the initial neural network model meets preset conditions, the training is stopped, and the initial neural network is used as the neural network model.

[0053] S105: Determine surgical 3D modeling printing parameters based on the output results.

[0054] Optionally, due to different surgical procedures, when the preset surgical plan is single-tooth implantation, the surgical 3D modeling and printing parameters are the implantation angle of the single missing tooth; when the preset surgical plan is multiple-tooth implantation, the surgical 3D modeling and printing parameters are the implantation angles of multiple missing teeth and the arrangement order of multiple missing teeth; when the preset surgical plan is maxillary sinus floor lift and immediate implantation, the surgical 3D modeling and printing parameters are safety thresholds, which include the cutting depth.

[0055] The embodiment of the present application proposes an oral 3D modeling and printing method based on a multimodal neural network. First, the target CT image data is obtained, and the oral tooth model data is obtained based on the target CT image data, wherein the target CT image data is CBCT voxel data, and then the target intraoral data is obtained, and the target intraoral data includes occlusal surface data, buccal data and lingual data, wherein the target intraoral data is oral scan point cloud data, and then the target facial data is obtained, and the target facial data is facial soft tissue data, and then the trained neural network model is obtained, and the oral tooth model data, the target intraoral data and the target facial data are imported into the neural network model, and the output result of the neural network model is obtained, and finally, the surgical 3D modeling and printing parameters are determined based on the output result. The embodiment of the present application proposes an oral 3D modeling and printing method based on a multimodal neural network, which determines the surgical 3D modeling and printing parameters by combining multiple data through a neural network model, significantly reduces the manual operation steps, and automatically completes the full process planning from data processing to solution generation through a deep learning model. The system can automatically identify key anatomical structures, calculate optimal implant parameters and generate execution files, greatly reducing the reliance on doctors' experience, significantly improving planning efficiency, and enabling doctors to devote more energy to program evaluation and optimization. Through multimodal data fusion technology, accurate registration and deep integration of CBCT, oral scan and facial scan data are achieved. The system uses advanced deep learning algorithms for feature extraction and analysis, which significantly improves the accuracy of anatomical structure recognition and effectively avoids the errors that may be caused by traditional manual measurement, thereby ensuring the reliability and accuracy of the planning plan. Taking full account of individual differences in patients, it can automatically generate personalized implant plans based on the anatomical characteristics, bone conditions and aesthetic needs of specific cases. The system optimizes the implant position, angle and depth through multi-dimensional analysis, and combines facial soft tissue data for aesthetic evaluation, realizing truly patient-specific planning.

[0056] Based on the same inventive concept, the embodiment of the present application provides an oral 3D modeling and printing system based on a multimodal neural network, which is configured as follows: Acquire target CT image data, and acquire oral tooth model data based on the target CT image data, wherein the target CT image data is CBCT voxel data; Acquire target intra-oral data, which includes occlusal surface data, buccal data, and lingual data, wherein the target intra-oral data is oral scan point cloud data; Acquire target facial data, where the target facial data is facial soft tissue data; Obtaining a trained neural network model, importing oral tooth model data, target intraoral data, and target facial data into the neural network model, and obtaining an output result of the neural network model; Based on the output results, the surgical 3D modeling printing parameters are determined.

[0057] In conjunction with the second aspect, in some embodiments, the system is configured as follows: Obtain preset surgical plans, including: single tooth implant, multiple tooth implant, maxillary sinus floor lift and immediate implant; Based on the output results, determine the surgical 3D modeling printing parameters, including: When the preset surgical plan is a single missing tooth implant, the surgical 3D modeling and printing parameters are the single missing tooth implant angle; When the preset surgical plan is to implant multiple missing teeth, the surgical 3D modeling and printing parameters are the implantation angles of the multiple missing teeth and the arrangement order of the multiple missing teeth; When the preset surgical plan is maxillary sinus floor lift and immediate implantation, the surgical 3D modeling and printing parameters are safety thresholds, which include the cutting depth.

[0058] In conjunction with the second aspect, in some embodiments, the system is configured as follows: Before obtaining the trained neural network model, importing the oral tooth model data, the target intraoral data and the target facial data into the neural network model, and obtaining the output result of the neural network model, the following steps are also included: A training set is obtained, and an initial neural network model is trained based on the training set. When the initial neural network model meets preset conditions, the training is stopped, and the initial neural network is used as the neural network model.

[0059] In conjunction with the second aspect, in some embodiments, the system is configured as follows: Acquire target CT image data, and acquire oral tooth model data based on the target CT image data, wherein the target CT image data is CBCT voxel data, including: Acquire a plurality of adjacent oral CT images, and determine an oral digital model based on the plurality of oral CT images; Determine the target area based on the oral digital model, wherein the target area is a partial model in the oral digital model, and the target area includes all teeth and gums; determining a first CT image, wherein a scanning surface for acquiring the first CT image interferes with the target area; Acquiring a second CT image, wherein a scanning plane for acquiring the second CT image and a scanning plane for acquiring the first CT image are adjacent scanning planes; Obtaining an average grayscale value Qv1 of the first CT image and an average grayscale value Qv2 of the second CT image; Segmenting the first CT image to form a plurality of first target images, and obtaining a gray value Q1 of each first target image, segmenting the second CT image to form a plurality of second target images, and obtaining a gray value Q2 of each second target image; Obtaining a grayscale difference H1 between a plurality of average grayscale values ​​Qv1 and grayscale value Q1, and obtaining a grayscale difference H2 between a plurality of average grayscale values ​​Qv2 and grayscale value Q2; Overlap the first target image and the second target image at the same position, and determine the tooth boundary according to the grayscale difference H1 and the grayscale difference H2; Determine the single tooth model based on the tooth boundaries.

[0060] In conjunction with the second aspect, in some embodiments, the system is configured as follows: Overlapping the first target image and the second target image at the same position, and determining the tooth boundary according to the grayscale difference H1 and the grayscale difference H2, including: Acquire the first target image and the second target image at the overlapping position, and determine the variable Ht between the grayscale difference H1 and the grayscale difference H2; Whether the overlapping position is a tooth boundary is determined based on the variable Ht.

[0061] In conjunction with the second aspect, in some embodiments, the system is configured as follows: Determining whether the overlapping position is a tooth boundary based on the variable Ht includes: Get the average variable h between the average gray value Qv1 and the average gray value Qv2; The variable Ht is compared with the average variable h. If the variable Ht is greater than the average variable h, it is determined that the tooth boundary is located between the scanning surface for acquiring the first target image and the scanning surface for acquiring the second target image.

[0062] In conjunction with the second aspect, in some embodiments, the system is configured as follows: The target area is determined based on the oral digital model, wherein the target area is a partial model in the oral digital model, and the target area includes all teeth and gums, including: Decompose the oral digital model into multiple sub-models; Determine the degree of adhesion between multiple sub-models based on fuzzy algorithm; Determining components of the oral digital model based on the adhesion degree, wherein sub-models with the same adhesion degree are one component; One or more components among the multiple components are determined as target areas through boundary recognition.

[0063] In conjunction with the second aspect, in some embodiments, the system is configured as follows: One or more components among the multiple components are determined as target areas through boundary recognition, including: Obtaining a preset model set, the preset model set including tooth model data and gum model data; Match the component with the preset model set and obtain the matching result; According to the matching results, the target area is determined.

[0064] The embodiment of the present application proposes an oral 3D modeling and printing system based on a multimodal neural network. First, the target CT image data is obtained, and the oral tooth model data is obtained based on the target CT image data, wherein the target CT image data is CBCT voxel data, and then the target intraoral data is obtained, and the target intraoral data includes occlusal surface data, buccal data and lingual data, wherein the target intraoral data is oral scan point cloud data, and then the target facial data is obtained, and the target facial data is facial soft tissue data, and then the trained neural network model is obtained, and the oral tooth model data, the target intraoral data and the target facial data are imported into the neural network model, and the output result of the neural network model is obtained, and finally, the surgical 3D modeling and printing parameters are determined based on the output result. The embodiment of the present application proposes an oral 3D modeling and printing method based on a multimodal neural network, which determines the surgical 3D modeling and printing parameters by combining a neural network model with multiple data, significantly reduces the manual operation steps, and automatically completes the full process planning from data processing to solution generation through a deep learning model. The system can automatically identify key anatomical structures, calculate optimal implant parameters and generate execution files, greatly reducing the reliance on doctors' experience, significantly improving planning efficiency, and enabling doctors to devote more energy to program evaluation and optimization. Through multimodal data fusion technology, accurate registration and deep integration of CBCT, oral scan and facial scan data are achieved. The system uses advanced deep learning algorithms for feature extraction and analysis, which significantly improves the accuracy of anatomical structure recognition and effectively avoids the errors that may be caused by traditional manual measurement, thereby ensuring the reliability and accuracy of the planning plan. Taking full account of individual differences in patients, it can automatically generate personalized implant plans based on the anatomical characteristics, bone conditions and aesthetic needs of specific cases. The system optimizes the implant position, angle and depth through multi-dimensional analysis, and combines facial soft tissue data for aesthetic evaluation, realizing truly patient-specific planning.

[0065] Based on the same inventive concept, an embodiment of the present application further proposes an electronic device, the electronic device comprising: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the oral 3D modeling and printing method based on a multimodal neural network according to an embodiment of the present application.

[0066] In addition, to achieve the above-mentioned purpose, an embodiment of the present application also proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the oral 3D modeling and printing method based on a multimodal neural network of the embodiment of the present application.

[0067] The following is a detailed introduction to the various components of electronic equipment: The processor is the control center of the electronic device, which can be a processor or a general term for multiple processing elements. For example, the processor is one or more central processing units (CPUs), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (FPGAs).

[0068] Optionally, the processor can perform various functions of the electronic device by running or executing a software program stored in the memory, and calling data stored in the memory.

[0069] The memory is used to store the software program for executing the solution of the present invention, and the execution is controlled by the processor. The specific implementation method can refer to the above method embodiment and will not be repeated here.

[0070] Optionally, the memory may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may be integrated with the processor, or may exist independently and be coupled to the processor through an interface circuit of the electronic device, which is not specifically limited in the embodiments of the present invention.

[0071] A transceiver is used to communicate with a network device or a terminal device.

[0072] Optionally, the transceiver may include a receiver and a transmitter, wherein the receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0073] Optionally, the transceiver may be integrated with the processor, or may exist independently and be coupled to the processor via an interface circuit of the router, which is not specifically limited in the embodiment of the present invention.

[0074] In addition, the technical effects of the electronic device can refer to the technical effects of the data transmission method in the above method embodiment, which will not be repeated here.

[0075] It should be understood that the processor in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0076] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0077] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When a computer instruction or computer program is loaded or executed on a computer, a process or function according to an embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0078] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0079] In the present invention, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0080] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0081] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

Claims

1. An oral 3D modeling and printing method based on a multimodal neural network, characterized in that: The method comprises: Acquire target CT image data, and acquire oral tooth model data based on the target CT image data, wherein the target CT image data is CBCT voxel data; Acquire target intra-oral data, wherein the target intra-oral data includes occlusal surface data, buccal data, and lingual data, wherein the target intra-oral data is oral scan point cloud data; Acquire target facial data, where the target facial data is facial soft tissue data; Obtaining a trained neural network model, importing the oral tooth model data, the target intraoral data and the target facial data into the neural network model, and obtaining an output result of the neural network model; Based on the output results, surgical 3D modeling printing parameters are determined.

2. The oral 3D modeling and printing method based on a multimodal neural network according to claim 1, characterized in that: Also includes: Obtaining a preset surgical plan, wherein the preset surgical plan includes: single tooth implantation, multiple tooth implantation, maxillary sinus floor lifting, and immediate implantation; Based on the output results, surgical 3D modeling printing parameters are determined, including: When the preset surgical plan is the single missing tooth implantation, the surgical 3D modeling printing parameter is the single missing tooth implantation angle; When the preset surgical plan is implantation of the multiple missing teeth, the surgical 3D modeling and printing parameters are the implantation angles of the multiple missing teeth and the arrangement order of the multiple missing teeth; When the preset surgical plan is the maxillary sinus floor lift and the immediate implant, the surgical 3D modeling and printing parameters are safety thresholds, and the safety thresholds include the cutting depth.

3. The oral 3D modeling and printing method based on a multimodal neural network according to claim 1, characterized in that: Before obtaining the trained neural network model, importing the oral tooth model data, the target intraoral data and the target facial data into the neural network model, and obtaining the output result of the neural network model, the method further includes: A training set is obtained, an initial neural network model is trained based on the training set, the training is stopped when the initial neural network model meets a preset condition, and the initial neural network is used as the neural network model.

4. The oral 3D modeling and printing method based on a multimodal neural network according to claim 1, characterized in that: Acquire target CT image data, and acquire oral tooth model data based on the target CT image data, wherein the target CT image data is CBCT voxel data, including: Acquire a plurality of adjacent oral CT images, and determine an oral digital model based on the plurality of oral CT images; Determine a target area based on the oral digital model, wherein the target area is a partial model in the oral digital model, and the target area includes all teeth and gums; determining a first CT image, wherein a scanning surface for acquiring the first CT image interferes with the target area; Acquire a second CT image, wherein a scanning plane for acquiring the second CT image and the scanning plane for acquiring the first CT image are adjacent scanning planes; Obtaining an average grayscale value Qv1 of the first CT image and an average grayscale value Qv2 of the second CT image; Segmenting the first CT image to form a plurality of first target images, and obtaining the gray value Q1 of each of the first target images, segmenting the second CT image to form a plurality of second target images, and obtaining the gray value Q2 of each of the second target images; Obtaining a plurality of grayscale differences H1 between the average grayscale value Qv1 and the grayscale value Q1, and obtaining a plurality of grayscale differences H2 between the average grayscale value Qv2 and the grayscale value Q2; Overlapping the first target image and the second target image at the same position, and determining the tooth boundary according to the grayscale difference H1 and the grayscale difference H2; Determine a single tooth model according to the tooth boundary; The tooth model data is acquired based on the tooth model.

5. The oral 3D modeling and printing method based on a multimodal neural network according to claim 4, characterized in that: Overlapping the first target image and the second target image at the same position, and determining the tooth boundary according to the grayscale difference H1 and the grayscale difference H2, comprises: Acquire the first target image and the second target image at overlapping positions, and determine a variable Ht between the grayscale difference H1 and the grayscale difference H2; It is determined whether the overlapping position is the tooth boundary based on the variable Ht.

6. The oral 3D modeling and printing method based on a multimodal neural network according to claim 5, characterized in that: Determining whether the overlapping position is the tooth boundary based on the variable Ht includes: Obtaining an average variable h between the average gray value Qv1 and the average gray value Qv2; The variable Ht is compared with the average variable h. If the variable Ht is greater than the average variable h, it is determined that the tooth boundary is located between the scanning surface for acquiring the first target image and the scanning surface for acquiring the second target image.

7. The oral 3D modeling and printing method based on a multimodal neural network according to claim 6, characterized in that: Determine a target area based on the oral digital model, wherein the target area is a partial model in the oral digital model, and the target area includes all teeth and gums, including: Decomposing the oral digital model into a plurality of sub-models; Determining the degree of adhesion between the plurality of sub-models based on a fuzzy algorithm; Determining components of the oral digital model based on the adhesion degree, wherein the sub-models having the same adhesion degree are one component; One or more components among the plurality of components are determined as the target area through boundary recognition.

8. The oral 3D modeling and printing method based on a multimodal neural network according to claim 7, characterized in that: Determining one or more components of the plurality of components as the target area through boundary recognition includes: Acquire a preset model set, wherein the preset model set includes tooth model data and gum model data; Matching the component with the preset model set and obtaining a matching result; The target area is determined according to the matching result.

9. An oral 3D modeling and printing system based on a multimodal neural network, characterized in that: The system is configured to: Acquire target CT image data, and acquire oral tooth model data based on the target CT image data, wherein the target CT image data is CBCT voxel data; Acquire target intra-oral data, wherein the target intra-oral data includes occlusal surface data, buccal data, and lingual data, wherein the target intra-oral data is oral scan point cloud data; Acquire target facial data, where the target facial data is facial soft tissue data; Obtaining a trained neural network model, importing the oral tooth model data, the target intraoral data and the target facial data into the neural network model, and obtaining an output result of the neural network model; Based on the output results, surgical 3D modeling printing parameters are determined.

10. An electronic device comprising a processor, a communication interface, a memory and a communication bus, wherein: The processor, the communication interface, and the memory communicate with each other via the communication bus; The memory is used to store computer programs; The processor is used to implement the method according to any one of claims 1 to 8 when executing the program stored in the memory.

Images