Oral health monitoring method and system based on three-dimensional reconstruction and multi-feature constraint

Abstract

The disclosure provides a method and system for oral health monitoring based on three-dimensional reconstruction and multi-feature constraint, relates to the technical field of medical data processing and oral monitoring, and comprises the following steps: acquiring multi-angle high-definition images and depth data in the oral cavity; performing semantic segmentation on the preprocessed multi-angle high-definition images to identify the tooth structure and key areas in the oral cavity; using an edge-guided depth reconstruction method to construct a three-dimensional oral cavity model; introducing a time stamp and mapping the depth data, tooth structure and key areas in the oral cavity to the three-dimensional oral cavity model; on the basis of the three-dimensional space dimension, increasing the time dimension and the key point distribution dimension to construct a multi-dimensional dynamic oral cavity model; based on the multi-dimensional dynamic oral cavity model, analyzing the evolution trend of the key areas through a spatio-temporal dynamic prediction network, predicting the future evolution risk, and obtaining a risk assessment result; generating a health management suggestion according to the risk assessment result and feeding back to the multi-dimensional dynamic oral cavity model to form a closed-loop feedback mechanism.

Description

Technical Field

[0001] The present disclosure relates to the technical fields of medical data processing and oral monitoring, and in particular to an oral health monitoring method and system based on three-dimensional reconstruction and multi-feature constraints. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.

[0003] Oral health is an important component of personal health. Early diagnosis and treatment of oral diseases directly impact a patient's quality of life and overall health. Oral diseases include caries, gingivitis, periodontal disease, and tartar. Most oral diseases are difficult to detect in their early stages and remain undetected until they have become more severe. Therefore, early detection and prevention of oral diseases are crucial. However, traditional oral health examinations rely primarily on manual inspection or two-dimensional imaging (such as X-rays, dental films, and panoramic films). These methods have limitations, particularly in terms of early warning of disease and detection of hidden lesions. Traditional oral examinations are typically based on the physician's experience, requiring the physician to identify oral diseases through manual inspection and analysis of imaging data. This process is easily influenced by the physician's experience, leading to a certain degree of subjectivity and the possibility of misdiagnosis.

[0004] Furthermore, traditional oral examination methods lack long-term, continuous data accumulation, making it impossible to monitor patients' oral health status in real time. For patients without obvious symptoms or early lesions, the lack of health records and dynamic tracking often leads to delayed detection of oral health problems.

[0005] In existing technologies, although some hospitals or clinics provide regular oral examinations, these examinations are usually sporadic and lack a holistic approach to long-term oral health monitoring, making it impossible to effectively assess changing trends in oral health. Currently, oral health management research based on image recognition and artificial intelligence attempts to improve the efficiency and accuracy of oral disease detection through automated analysis, but the following technical issues still exist:

[0006] 1) Existing technologies mainly focus on the processing and analysis of single-shot imaging data, and lack the collection and analysis of long-term oral health data.

[0007] 2) Oral health management lacks complete long-term health records, making it impossible to carry out predictive intervention for diseases.

[0008] 3) The existing oral disease detection system mainly relies on hospitals and professional equipment. Ordinary users have weak ability to monitor themselves at home, which limits the popularization and application of oral health management technology. Summary of the Invention

[0009] In order to solve the above problems, the present disclosure proposes an oral health monitoring method and system based on three-dimensional reconstruction and multi-feature constraints. By combining three-dimensional reconstruction technology with deep learning technology of time series analysis, long-term dynamic monitoring of oral health and disease risk prediction can be achieved, and a complete closed-loop mechanism is realized from user self-examination, data upload and analysis, health risk prediction, generation of health management suggestions, to data collection again after the user executes the suggestions and feedback model optimization.

[0010] According to some embodiments, the present disclosure adopts the following technical solutions:

[0011] Oral health monitoring methods based on 3D reconstruction and multi-feature constraints include:

[0012] Acquire multi-angle high-definition images and depth data of the oral cavity and pre-process them;

[0013] Semantic segmentation is performed on pre-processed multi-angle high-definition images using an adaptive spatiotemporal fusion semantic segmentation network. Guided feature maps, including the image's color gradient map and depth edge map, are introduced as constraints to guide the network to focus on discriminative features at tissue boundaries to identify dental structures and key areas within the oral cavity.

[0014] The edge-guided deep reconstruction method is used to extract key feature points from multi-angle high-definition images, generate a sparse point cloud of the oral structure, generate a dense point cloud based on the sparse point cloud, and use the surface fitting method to construct a 3D oral model.

[0015] Introducing timestamps, and mapping depth data, oral tooth structures, and key areas to a 3D oral model. Adding time dimensions and key point distribution dimensions to the 3D spatial dimension, a multi-dimensional dynamic oral model is constructed.

[0016] Based on the multi-dimensional dynamic oral model, the evolution trends of key areas are analyzed through the spatiotemporal dynamic prediction network, and future evolution risks are predicted to obtain risk assessment results. Health management recommendations are generated based on the risk assessment results and fed back to the multi-dimensional dynamic oral model to form a closed-loop feedback mechanism.

[0017] According to some embodiments, the present disclosure adopts the following technical solutions:

[0018] Oral health monitoring system based on 3D reconstruction and multi-feature constraints, including:

[0019] Data acquisition module, used to obtain multi-angle high-definition images and depth data of the oral cavity and pre-process them;

[0020] The semantic recognition module is configured to perform semantic segmentation on the preprocessed multi-angle high-definition image by using an adaptive space-time fusion semantic segmentation network, and introduce a guided feature map as a constraint condition, including a color gradient map and a depth edge map of the image, to guide the network to focus on identifying discriminative features at a tissue boundary, recognize intraoral dental structures, and key regions.

[0021] The model construction module is configured to extract key feature points of the multi-angle high-definition image by using an edge-guided depth reconstruction method, generate a sparse point cloud of the oral structure, generate a dense point cloud based on the sparse point cloud, and construct a three-dimensional oral model by using a surface fitting method.

[0022] The dynamic analysis module is configured to introduce a time stamp, and map the depth data, the intraoral dental structures, and the key regions to the three-dimensional oral model, increase a time dimension and a key point distribution dimension on the basis of a three-dimensional spatial dimension, and construct a multi-dimensional dynamic oral model.

[0023] The risk prediction module is configured to analyze an evolution trend of the key regions by using a space-time dynamic prediction network based on the multi-dimensional dynamic oral model, predict a future evolution risk, and obtain a risk assessment result; generate a health management suggestion according to the risk assessment result, and feed back to the multi-dimensional dynamic oral model to form a closed-loop feedback mechanism.

[0024] According to some embodiments, the present disclosure adopts the following technical solution:

[0025] A computer program product includes a computer program, which, when executed by a processor, implements the oral health monitoring method based on three-dimensional reconstruction and multi-feature constraint.

[0026] According to some embodiments, the present disclosure adopts the following technical solution:

[0027] A non-transitory computer-readable storage medium is configured to store computer instructions, which, when executed by a processor, implement the oral health monitoring method based on three-dimensional reconstruction and multi-feature constraint.

[0028] According to some embodiments, the present disclosure adopts the following technical solution:

[0029] An electronic device includes a processor, a memory, and a computer program; the processor is connected with the memory, and the computer program is stored in the memory; when the electronic device is running, the processor executes the computer program stored in the memory, so that the electronic device executes the oral health monitoring method based on three-dimensional reconstruction and multi-feature constraint.

[0030] Compared with the prior art, the present disclosure has the following beneficial effects:

[0031] This oral health monitoring method, based on 3D reconstruction and multi-feature constraints, uses an adaptive spatiotemporal fusion semantic segmentation network to perform semantic segmentation on preprocessed multi-angle high-definition images, identifying dental structures and key areas within the oral cavity. This method addresses the semantic recognition of teeth, gums, and lesions within oral images, taking into account both spatial and temporal dynamics. The network utilizes spatial texture, temporal sequence, image edge, and depth features to enhance the perception of boundary regions and detail changes, improving segmentation accuracy and the stability of structure recognition.

[0032] The oral health monitoring method based on three-dimensional reconstruction and multi-feature constraints disclosed in the present invention uses an edge-guided depth reconstruction method algorithm to improve the accuracy of feature point extraction and the structural stability of the reconstructed model during the three-dimensional reconstruction process. This method combines the color texture boundaries of the image with the structural contours of the depth map to achieve high robustness and multi-view fusion extraction of key feature points. The key feature points of multi-angle high-definition images are extracted to generate a sparse point cloud of the oral structure. In order to improve the density and continuity of the point cloud, depth-guided disparity estimation is performed between the registered image pairs. Structural response constraints are introduced in this process to limit the disparity jump to only occur at significant fusion boundaries, thereby retaining key details in high-change areas such as the lesion edge and the tooth occlusal surface. The generated dense disparity map is fused with spatial consistency to reconstruct a dense point cloud. A surface fitting strategy using constrained smoothing is used to complete the triangular mesh reconstruction. The algorithm optimizes the overall surface structure by minimizing normal consistency and boundary contour tension, generating a three-dimensional oral model with continuous structure and clear details. It can fully present the tooth contour, gum undulations and lesion surface morphology to construct a three-dimensional oral model; with the help of deep learning and artificial intelligence technology, it greatly improves the early diagnosis rate and accuracy of oral diseases, reduces the subjectivity of traditional examinations that rely on doctors' experience, and significantly improves the efficiency and quality of diagnosis.

[0033] The disclosed oral health monitoring method based on three-dimensional reconstruction and multi-feature constraints stores the depth data, dental structures, and key areas of the oral cavity acquired each time in a cloud database, and performs version control through timestamps to form a dynamically updated 5D health profile. This profile adds time dimensions and key point distribution dimensions to the three-dimensional spatial dimension, constructing a multi-dimensional dynamic oral model, thereby achieving multi-dimensional integrated management of oral health information. By establishing a 5D health profile, doctors are also provided with personalized oral disease diagnosis and treatment plans, further implementing precision medicine in the field of oral health.

[0034] The oral health monitoring method based on three-dimensional reconstruction and multi-feature constraint of the present disclosure adopts a temporal-spatial dynamic prediction network (TSDP-Net) to uniformly model and predict the evolution trend of key areas of the oral cavity; generates health management suggestions according to the risk assessment results, and feeds back the analysis results to the dynamic model to form a closed-loop optimization mechanism, and provides long-term oral data health monitoring. This method is especially suitable for personal user self-checking and hospital professional diagnosis of oral health management scenarios, and can realize long-term monitoring and risk prediction of oral health. BRIEF DESCRIPTION OF DRAWINGS

[0035] The accompanying drawings, which form a part of this disclosure, are intended to provide further understanding of the present disclosure and are incorporated herein in their entirety. The schematic embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure.

[0036] Figure 1 A 5D oral health record structure diagram of an embodiment of the present disclosure;

[0037] Figure 2 A risk prediction flowchart of an embodiment of the present disclosure;

[0038] Figure 3 A health management closed-loop feedback flowchart of an embodiment of the present disclosure. DETAILED DESCRIPTION

[0039] The present disclosure will be further described below in conjunction with the accompanying drawings and embodiments.

[0040] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present disclosure belongs.

[0041] It should be noted that the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the exemplary embodiments according to the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form, and in addition, it should be understood that when the terms "comprise" and / or "include" are used in the present specification, they indicate the presence of a feature, step, operation, device, component and / or combination thereof.

[0042] Embodiment 1

[0043] In an embodiment of the present disclosure, an oral health monitoring method based on three-dimensional reconstruction and multi-feature constraint is provided. This method is especially suitable for personal user self-checking and hospital professional diagnosis of oral health management scenarios, and can realize long-term monitoring and risk prediction of oral health. The steps are as follows:

[0044] Step one: Obtain multi-angle high-definition images and depth data in the oral cavity and pre-process them;

[0045] Step two: Perform semantic segmentation on the pre-processed multi-angle high-definition images using an adaptive spatio-temporal fusion semantic segmentation network, and introduce guided feature maps as constraint conditions, including color gradient maps and depth edge maps of the images, to guide the network to focus on discriminative features at tissue boundaries to identify dental structures and key areas in the oral cavity;

[0046] Step three: Extract key feature points of multi-angle high-definition images using an edge-guided depth reconstruction method, generate sparse point clouds of oral cavity structures, generate dense point clouds based on sparse point clouds, and construct a three-dimensional oral cavity model using a surface fitting method;

[0047] Step four: Introduce a timestamp and map depth data, dental structures and key areas in the oral cavity to the three-dimensional oral cavity model, increase the time dimension and key point distribution dimension based on the three-dimensional spatial dimension, and construct a multi-dimensional dynamic oral cavity model;

[0048] Step five: Based on the multi-dimensional dynamic oral cavity model, analyze the evolution trend of the key area through a mixture model, predict the future evolution risk, and obtain the risk assessment result; generate health management suggestions according to the risk assessment result, and feed back to the multi-dimensional dynamic oral cavity model to form a closed-loop feedback mechanism.

[0049] As an embodiment, the specific implementation process of a three-dimensional reconstruction and multi-feature constraint-based oral health monitoring method of the present disclosure is as follows:

[0050] Step 1: Obtain multi-angle high-definition images and depth data in the oral cavity and pre-process them;

[0051] Specifically, oral health data is collected through intelligent terminal devices such as smartphones, tablets, etc. The device uses its built-in high-definition camera, accelerometer, gyroscope, and depth sensor based on the time-of-flight (ToF) principle to obtain high-definition images and depth information in the oral cavity. The ToF depth sensor obtains the depth value of each pixel by emitting an infrared light pulse and measuring the round-trip time or phase shift of the light signal. The ToF depth information is not only used to accurately construct a three-dimensional oral cavity model, but also assists in multi-view image registration, improves semantic segmentation accuracy, and lesion positioning, etc. The user performs oral cavity internal shooting under the guidance of the device, and the system automatically records the timestamp and spatial position information of the shooting, thereby ensuring the accuracy and traceability of the data.

[0052] During data collection, the device will prompt the user to take images of the inside of the mouth at different angles and positions, capturing high-definition images from multiple angles. By combining these images with depth data, the information obtained is comprehensive and accurate. The data captured by the user is uploaded to the cloud server in real time for subsequent processing and analysis.

[0053] Further, after data collection, the multi-angle high-definition images and depth information are transmitted to the cloud server for further optimization processing. The role of the data preprocessing module is to perform necessary optimization processing on the original images and depth data to ensure the accuracy and precision of subsequent analysis. Specifically, the multi-angle high-definition image preprocessing process includes image noise reduction, white balance correction, illumination compensation, and format standardization processing operations. Image noise reduction improves image clarity by removing noise in the image; illumination compensation and white balance correction ensure that the image maintains good visual effects under different shooting environments.

[0054] In addition, the depth information is optimized by depth map generation and optimization, and the specific process is as follows:

[0055] The initial depth information of the user's oral scene is captured in real time using a ToF (Time-of-Flight) depth sensor. This sensor calculates the distance between each pixel point and the sensor by emitting modulated infrared light and receiving its reflected signal, thereby obtaining the original depth map (depth information) of the entire oral region. To improve the quality of the depth map and reduce data anomalies caused by hardware noise, light interference, or surface reflections, further optimization processing is performed on the depth map using a guided filter (Guided Filter) algorithm for edge-preserving smoothing. This algorithm uses the corresponding RGB image as a guide image, assuming a linear relationship between the output depth value and the guide image brightness within each local window, and solving the linear coefficient through least squares method to generate a smooth and structurally consistent optimized depth map. This processing not only effectively removes noise and artifacts in the depth map, but also preserves the depth mutation characteristics of structures such as tooth edges and gum margins, significantly improving the accuracy and stability of subsequent three-dimensional point cloud reconstruction. The design of this module ensures that the image data has good processing quality before being input into the subsequent model, and provides a stable and reliable input data basis.

[0056] Step 2: Perform semantic segmentation on the preprocessed multi-angle high-definition images to identify the tooth structure and key areas within the mouth, as follows:

[0057] To improve the segmentation accuracy and the stability of structure recognition, the disclosure proposes an adaptive spatial-temporal hybrid semantic segmentation network (ASTH-Net) for the semantic recognition task of teeth, gums and lesion regions in oral images, which takes into account spatial features and temporal dynamic information. The network comprehensively utilizes spatial texture, time series, image edges and depth features to enhance the perception ability of boundary regions and detail changes.

[0058] Specifically, the adaptive spatial-temporal hybrid semantic segmentation network (ASTH-Net) includes two main branches in structure, which are spatial feature extraction branch and time information fusion branch, respectively. The spatial feature extraction branch is used to extract local structure information of teeth and soft tissues from single frame images to obtain spatial features The time information fusion branch aggregates the evolution information between adjacent frames through a lightweight ConvLSTM module to obtain time features ;

[0059] To further improve the accuracy of boundary detection, guided feature maps are introduced as constraint conditions, including color gradient maps and depth edge maps (processed from ToF depth maps) to guide the network to focus on the discriminative features at the tissue boundary, and to realize the fusion of spatial features and time features. Specifically as follows:

[0060] In the feature fusion stage, a guided weighted fusion mechanism is constructed. Let the spatial features be , the time features be , and the guided weight be , and the final fused features be defined as follows:

[0061]

[0062] Wherein, The value of is obtained based on the color gradient intensity and the depth boundary amplitude . Specifically, within each local receptive field, the color channel gradient intensity and the depth boundary response are calculated, and the fusion weight is generated by weighted normalization:

[0063]

[0064] Wherein, , are the adjustment coefficients of color and depth guide terms, respectively, is a numerical stability term. The weight presents a higher value in the area where the boundary changes dramatically, enhancing the network's response ability to the edge structure; in the smooth area, it reduces the response to avoid excessive sensitivity.

[0065] After training and optimization, ASTH-Net outputs semantic segmentation results in the decoding stage, automatically identifying the main structure and potential key area (lesion area) in the oral image.

[0066] As an embodiment, the specific implementation process in the adaptive spatio-temporal fusion semantic segmentation network is as follows:

[0067] Step 1) input high-definition image;

[0068] Step 2) the spatial feature extraction branch adopts a multi-scale dilated convolution network, combining the texture details and morphological edges in the image to extract static structural features ; the time information fusion branch adopts a lightweight ConvLSTM module, receiving adjacent multi-frame sequence images and modeling their dynamic evolution trend to output time sequence features . Among them, all the convolution kernel parameters are dynamically adjusted in size and shape based on local guided information (color gradient ) in the training stage, so that the network automatically adapts to the structural complexity in different oral areas.

[0069] Step 3) feature fusion and boundary guided optimization: in the fusion stage, a guide map based on color gradient and depth boundary is introduced to participate in weight calculation, by analyzing the color and depth change rate in the 5x5 neighborhood around each pixel, the response activation of gum margin, occlusal surface, and dental gap is strengthened, and the consistency and clarity of the segmentation edge are improved.

[0070] Step 4) key area output (lesion category): based on the fused features, the system outputs the semantic class probability map of each pixel through the full convolution prediction head (Segmentation Head). The lesion category covers common dental lesion areas (such as dental caries, dental calculus, and gingivitis), and realizes automatic labeling in a pixel-level manner. To improve the positioning accuracy, the lesion prediction module introduces a regional attention mechanism, dynamically enhances the response weight of the area where the lesion may exist by analyzing the local activation pattern in the fused features, thereby suppressing false detection in non-target areas. The lesion judgment also combines structure prior information (such as tooth surface normal direction and gum relative position) and historical record comparison to further filter short-term abnormal signals, ensuring that the output lesion labeling is stable and has temporal consistency.

[0071] Step 5) the labeling output result includes the pixel mask of the lesion area and the area contour coordinates of each type of lesion;

[0072] Step 6) Structural positioning index (corresponding to the key point number of the 3D model): mark the confidence score (0-1) for subsequent risk assessment weighting processing.

[0073] Through the above process, high-resolution and high-boundary consistency segmentation performance can be maintained in oral images with complex texture structures and individual differences, and reliable structural input can be provided for dynamic health modeling.

[0074] Step 3: Use edge-guided deep reconstruction to extract key feature points from multi-angle HD images, generate a sparse point cloud of the oral structure, generate a dense point cloud based on the sparse point cloud, and construct a 3D oral model using a surface fitting method.

[0075] Specifically, this module proposes an edge-guided depth reconstruction method (EGD-Recon) to improve the accuracy of feature point extraction and the structural stability of the reconstructed model during 3D reconstruction. This method combines the color and texture boundaries of the image with the structural contours of the depth map to achieve highly robust, multi-view fusion extraction of key feature points.

[0076] In practice, a color gradient map and depth response map are first constructed for each captured image frame. The color gradient map describes areas of texture abrupt changes on the tooth surface, while the depth response map reflects the geometric boundaries of areas like the gums and interdental spaces. The system then fuses these two maps to generate a candidate feature map. Within this map, points with high boundary consistency and significant local variations are adaptively selected as key feature points.

[0077] The registration between feature points uses a joint matching cost function based on regional matching confidence and boundary similarity to avoid mismatches caused by purely geometric or purely texture features. On this basis, the system builds a sparse point cloud representation of the three-dimensional topological skeleton of the oral structure.

[0078] To improve point cloud density and continuity, EGD-Recon then performs depth-guided disparity estimation between the registered image pairs. This process introduces structural response constraints, restricting disparity jumps to significant areas of fusion boundaries. This preserves critical details in high-variance areas such as lesion edges and tooth occlusal surfaces. The resulting dense disparity maps are then spatially consistent fused to reconstruct a dense point cloud.

[0079] Based on the point cloud, a constrained smooth surface fitting strategy is used to reconstruct the triangulated mesh. This algorithm optimizes the overall surface structure by minimizing normal consistency and boundary contour tension, generating a structurally continuous and detailed 3D oral model that fully captures tooth contours, gingival contours, and lesion surface morphology.

[0080] Ultimately, the module outputs a standardized, high-fidelity three-dimensional oral model, and supports subsequent alignment and analysis of lesion dynamic changes in time series analysis.

[0081] Step 4: Introduce timestamps and map the depth map, oral tooth structure, and key areas to the 3D oral model. Add the time dimension and key point distribution dimension to the 3D spatial dimension to construct a multi-dimensional dynamic oral model.

[0082] Specifically, each user-captured data (including images, 3D models, lesion annotation information, etc.) is stored in a cloud database and versioned using timestamps, forming a dynamically updated 5D health record. This record builds upon the 3D spatial dimension by adding a time dimension (representing the time of each capture and model update) and a key distribution dimension (recording the type and location of each detected lesion). As each examination progresses, the new 3D model is continuously linked to the historical model via timestamps, constructing a multidimensional dynamic oral model encompassing multiple dimensions of 3D space, time, and lesion distribution, enabling multidimensional integrated management of oral health information.

[0083] Step 5: Based on the multidimensional dynamic oral model, analyze the evolution trend of key areas through the spatiotemporal dynamic prediction network, predict future evolution risks, and obtain risk assessment results; generate health management recommendations based on the risk assessment results and feed them back to the multidimensional dynamic oral model to form a closed-loop feedback mechanism.

[0084] Specifically, based on a multi-dimensional dynamic oral model, the Temporal-SpatialDynamic Prediction Network (TSDP-Net) is used to uniformly model the evolution trends and predict risks of key oral areas. Health management recommendations are generated based on the risk assessment results, and the analysis results are fed back into the dynamic model to form a closed-loop optimization mechanism.

[0085] Specifically, the TSDP-Net network consists of a spatial perception module, a temporal evolution encoding module, and a fusion prediction module. The network input is a user's historical 5D oral archive sequence (including RGB images, depth maps, structural annotations, and risk distribution maps). The output is a sequence of key area risk scores for several future cycles, as well as the types and spatial trends of possible lesion development.

[0086] Furthermore, the spatial perception module is used to extract static spatial features of teeth, gums, and key areas in the current image. An improved convolutional component is used to extract multi-channel indicators such as boundary clarity, structural complexity, and lesion density. To enhance the ability to identify boundary structures, an edge direction tensor is embedded in the feature extraction process. Its calculation formula is as follows:

[0087]

[0088] in, and are the gradients of the image in the horizontal and vertical directions, is the main direction of the current area,

[0089] 𝐼 is the input image. This design is used to enhance the response sensitivity of the lesion edge.

[0090] The time evolution encoding module is used to encode the trend information of the user's oral health changes over time. The module input is the structural change description vector of the key area at multiple moments and the risk state label, and through the state transition function Perform latent variable evolution modeling:

[0091]

[0092] in, For customized time memory units, the internal state uses gated transformation to process the weights of different time intervals, thereby distinguishing the development trends of acute and chronic lesions.

[0093] Fusion prediction module in spatial features and time evolution state Constructing a multidimensional fusion tensor based on The fusion process uses a dynamic attention weighting mechanism:

[0094] ,

[0095] in, is the Sigmoid activation function, is a trainable parameter. This mechanism can dynamically adjust the contribution of different dimensions to the final prediction.

[0096] Based on the above output results, the system generates a health management plan for users, including oral hygiene recommendations, dietary guidance, review cycles, and intervention points. It also continuously receives newly uploaded data for online updates of the TSDP-Net model, thereby realizing an individualized and dynamically optimized health management closed loop.

[0097] As an embodiment, the workflow and implementation effect of the oral health monitoring method based on three-dimensional reconstruction and time series disclosed in the present invention are as follows:

[0098] The present disclosure can provide home users with a convenient oral self-examination and health management program. Users collect oral images through smart devices, perform care according to the system's recommendations, and upload data to the cloud to form a 5D health profile. The system analyzes historical health data, monitors oral changes in real time, and issues warnings when health is abnormal, helping users take preventive measures to avoid serious oral problems. In addition, users can receive personalized care recommendations and adjust their diet and lifestyle habits based on health changes. The system reduces the need for users to visit the doctor regularly, reduces medical costs, and improves the efficiency and accuracy of oral health management. Through long-term data accumulation, users can detect potential problems in advance and obtain timely intervention.

[0099] As an example, in medical institutions, the present disclosure serves as a health management aid, providing more efficient and accurate oral health examinations and disease prediction. Doctors can access patients' long-term health records through the system and understand their health trends in real time. The system automatically detects lesions using deep learning algorithms, assisting doctors in developing personalized treatment plans. For example, it can provide timely notification of the location and development trends of caries or periodontitis. The system also assesses future disease risks based on historical data, providing patients with personalized health management recommendations and improving treatment efficiency and quality.

[0100] The system can also be widely used for large-scale oral health monitoring. Governments, communities, or medical institutions can use it to provide regular assessments for large populations. By analyzing oral health data, the system helps identify trends in oral disease prevalence, uncover potential public health issues, and enable timely intervention. Furthermore, through big data analysis, the system helps public health departments formulate more precise oral health policies and public health measures, thereby improving the efficiency of public health management.

[0101] Example 2

[0102] In one embodiment of the present disclosure, an oral health monitoring system based on three-dimensional reconstruction and multi-feature constraints is provided, comprising:

[0103] Data acquisition module, used to obtain multi-angle high-definition images and depth data of the oral cavity and pre-process them;

[0104] The semantic recognition module is used to perform semantic segmentation on pre-processed multi-angle high-definition images using an adaptive spatiotemporal fusion semantic segmentation network. It introduces guided feature maps as constraints, including the image's color gradient map and depth edge map, to guide the network to focus on discriminative features at tissue boundaries to identify dental structures and key areas within the oral cavity.

[0105] The model building module is used to extract key feature points from multi-angle high-definition images using edge-guided deep reconstruction methods, generate a sparse point cloud of the oral structure, generate a dense point cloud based on the sparse point cloud, and construct a 3D oral model using a surface fitting method;

[0106] The dynamic analysis module is used to introduce timestamps and map depth data, dental structures, and key areas in the mouth to a 3D oral model. Based on the 3D spatial dimension, the time dimension and key point distribution dimension are added to construct a multi-dimensional dynamic oral model.

[0107] The risk prediction module is used to analyze the evolution trends of key areas based on the multi-dimensional dynamic oral model through the spatiotemporal dynamic prediction network, predict future evolution risks, and obtain risk assessment results; generate health management recommendations based on the risk assessment results, and feed them back to the multi-dimensional dynamic oral model to form a closed-loop feedback mechanism.

[0108] Example 3

[0109] In one embodiment of the present disclosure, a computer program product is provided, including a computer program, which implements the oral health monitoring method based on three-dimensional reconstruction and multi-feature constraints when executed by a processor.

[0110] Example 4

[0111] In one embodiment of the present disclosure, a non-transitory computer-readable storage medium is provided, which is used to store computer instructions. When the computer instructions are executed by a processor, the oral health monitoring method based on three-dimensional reconstruction and multi-feature constraints is implemented.

[0112] Example 5

[0113] In one embodiment of the present disclosure, an electronic device is provided, comprising: a processor, a memory, and a computer program; wherein the processor is connected to the memory, and the computer program is stored in the memory. When the electronic device is running, the processor executes the computer program stored in the memory, so that the electronic device executes the oral health monitoring method based on three-dimensional reconstruction and multi-feature constraints.

[0114] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0115] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0116] Although the above describes the specific implementation methods of the present disclosure in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present disclosure. Those skilled in the art should understand that on the basis of the technical solution of the present disclosure, various modifications or variations that can be made by those skilled in the art without creative work are still within the scope of protection of the present disclosure.

Claims

1. Oral health monitoring method based on three-dimensional reconstruction and multi-feature constraints, characterized by: include: Acquire multi-angle high-definition images and depth data of the oral cavity, and perform pre-processing and optimization; The pre-processed multi-angle high-definition images are semantically segmented using an adaptive spatiotemporal fusion semantic segmentation network. Guided feature maps, including the image's color gradient map and depth edge map, are introduced as constraints to guide the network to focus on the discriminant features at tissue boundaries to identify dental structures and key areas within the oral cavity. The pre-processed multi-angle high-definition images are semantically segmented using an adaptive spatiotemporal fusion semantic segmentation network. The spatial feature extraction branch of the adaptive spatiotemporal fusion semantic segmentation network extracts local structural information of the tooth and soft tissue from a single frame image. The temporal information fusion branch aggregates the evolutionary information between adjacent frames through a lightweight ConvLSTM module. Guided feature maps, including the image's color gradient map and optimized depth map, are introduced as constraints to guide the network to focus on the discriminant features at tissue boundaries to identify various structures within the oral cavity, including teeth, gums, and potential key areas. An edge-guided deep reconstruction method is used to extract key feature points from multi-angle high-definition images, generate a sparse point cloud of the oral structure, generate a dense point cloud based on the sparse point cloud, and use a surface fitting method to construct a three-dimensional oral model. Among them, the edge-guided deep reconstruction method is used to combine the color texture boundary of the image with the structural contour of the depth map to realize the fusion extraction of key feature points and realize the three-dimensional reconstruction of the oral image. First, the color gradient map and depth response map of each frame of the high-definition image are obtained, and the two are fused to generate a candidate feature map. Points with high boundary consistency and obvious local changes are adaptively selected in the map as key feature points. A joint matching cost function is constructed based on the regional matching confidence and boundary similarity. A three-dimensional topological skeleton of the oral structure is established by the sparse point cloud. Depth-guided disparity estimation is performed between the registered image pairs. Structural response constraints are introduced to reconstruct a dense point cloud. The constrained smooth surface fitting strategy is used to complete the triangular mesh reconstruction. The overall surface structure is optimized by minimizing the normal consistency and boundary contour tension to generate a three-dimensional oral model. Introducing timestamps, and mapping depth data, oral tooth structures, and key areas to a 3D oral model. Adding time dimensions and key point distribution dimensions to the 3D spatial dimension, a multi-dimensional dynamic oral model is constructed. Based on a multi-dimensional dynamic oral model, the evolution trend of key areas is analyzed through a spatiotemporal dynamic prediction network to predict future evolution risks and obtain risk assessment results; Health management recommendations are generated based on the risk assessment results and fed back to the multi-dimensional dynamic oral model to form a closed-loop feedback mechanism.

2. The oral health monitoring method based on three-dimensional reconstruction and multi-feature constraints according to claim 1, characterized in that: High-definition cameras, accelerometers, gyroscopes, and time-of-flight (ToF) depth sensors are used to acquire high-definition images and depth information from the oral cavity. The distance between each pixel and the sensor is calculated by emitting modulated infrared light and receiving its reflected signal, thereby obtaining the original depth map of the entire oral area. A guided filtering algorithm is then used to optimize the original depth map, using the corresponding RGB image as the guidance image. A linear relationship is assumed between the output depth value and the brightness of the guidance image in each local window, and the linear coefficient is solved using the least squares method to generate a smooth and structurally consistent optimized depth map.

3. The oral health monitoring method based on three-dimensional reconstruction and multi-feature constraints according to claim 1, characterized in that: The preprocessing process includes image noise reduction, white balance correction, illumination compensation and format standardization.

4. The oral health monitoring method based on three-dimensional reconstruction and multi-feature constraints according to claim 1, characterized in that: The depth data, dental structures and key areas obtained each time are stored in a cloud database and version controlled by timestamps to form a dynamically updated 5D health file. Based on the three-dimensional space dimension, this file adds the time dimension and key point distribution dimension, and continuously associates with the historical model through timestamps to construct a multi-dimensional dynamic oral model that includes three-dimensional space, time and lesion distribution, thereby realizing multi-dimensional integrated management of oral health information.

5. The oral health monitoring method based on three-dimensional reconstruction and multi-feature constraints according to claim 1, characterized in that: Based on a multi-dimensional dynamic oral model, a spatiotemporal dynamic prediction network is used to uniformly model and predict the evolution trends of key oral regions; Health management recommendations are generated based on the risk assessment results, and the analysis results are fed back into the dynamic model to form a closed-loop optimization mechanism. The spatiotemporal dynamic prediction network consists of a spatial perception module, a time evolution coding module, and a fusion prediction module. The input is the user's historical 5D oral archive sequence, and the output is a sequence of key area risk scores for several future cycles, as well as the types and spatial trends of possible lesion development.

6. An oral health monitoring system based on three-dimensional reconstruction and multi-feature constraints, specifically implementing the oral health monitoring method based on three-dimensional reconstruction and multi-feature constraints according to any one of claims 1 to 5, characterized in that: include: Data acquisition module, used to obtain multi-angle high-definition images and depth data of the oral cavity and pre-process them; The semantic recognition module is used to perform semantic segmentation on pre-processed multi-angle high-definition images using an adaptive spatiotemporal fusion semantic segmentation network. It introduces guided feature maps as constraints, including the image's color gradient map and depth edge map, to guide the network to focus on discriminative features at tissue boundaries to identify dental structures and key areas within the oral cavity. The model building module is used to extract key feature points from multi-angle high-definition images using edge-guided deep reconstruction methods, generate a sparse point cloud of the oral structure, generate a dense point cloud based on the sparse point cloud, and construct a 3D oral model using a surface fitting method; The dynamic analysis module is used to introduce timestamps and map depth data, dental structures, and key areas in the mouth to a 3D oral model. Based on the 3D spatial dimension, the time dimension and key point distribution dimension are added to construct a multi-dimensional dynamic oral model. The risk prediction module is used to analyze the evolution trends of key areas based on the multi-dimensional dynamic oral model through the spatiotemporal dynamic prediction network, predict future evolution risks, and obtain risk assessment results; generate health management recommendations based on the risk assessment results, and feed them back to the multi-dimensional dynamic oral model to form a closed-loop feedback mechanism.

7. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the oral health monitoring method based on three-dimensional reconstruction and multi-feature constraints according to any one of claims 1 to 5 is implemented.

8. A non-transitory computer-readable storage medium, characterized in that The non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by the processor, the oral health monitoring method based on three-dimensional reconstruction and multi-feature constraints as described in any one of claims 1 to 5 is implemented.

9. An electronic device, characterized in that: include: A processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to implement the oral health monitoring method based on three-dimensional reconstruction and multi-feature constraints as described in any one of claims 1 to 5.

Images