Generation method of digital jaw pad based on navigation of joint intelligent interaction system

Abstract

The invention provides a method for generating a digital jaw pad based on joint intelligent interaction system navigation, which comprises the following steps of: acquiring multi-modal data of a patient and inputting the multi-modal data into a multi-modal fusion model to generate a digital dental three-dimensional model and an analysis result, iteratively adjusting the position of the lower jawbone of the digital dental three-dimensional model to a treatment position by adopting a multi-objective optimization algorithm; processing the port scanning STL data through an intelligent design engine to obtain an undercut area; the digital jaw pad is manufactured based on the position and the undercut area of the lower jawbone of the treatment position. The high-precision digital jaw pad is generated based on navigation of a full-process closed-loop joint intelligent interaction system fusing multi-modal data, dynamic evaluation, intelligent design and precise preparation.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of digital jaw, in particular to a method for generating a digital jaw pad based on intelligent interaction system navigation of joints. BACKGROUND

[0002] In the clinical treatment of temporomandibular joint disorder (TMD), the jaw pad as the core orthodontic appliance directly affects the treatment effect with its manufacturing precision and diagnosis and treatment process.

[0003] The current mainstream traditional jaw pad manufacturing adopts manual molding combined with empirical jaw adjustment scheme, that is, the jaw position is obtained by relying on the doctor's manual operation such as Gothic arch tracing method, and the condyle position is determined by experience, with an error often exceeding 0.5 mm, which often leads to the fact that the jaw pad cannot accurately reset the disc, and in the process of manufacturing the jaw pad, manual stacking of self-curing resin or 3D printing of preformed jaw pads needs to be repeatedly adjusted and ground to match, which takes 5-7 days, the edge precision is only ±0.5 mm, and about 30% of the cases need 3 or more rechecks for trial wearing to achieve the ideal fit.

[0004] Even though some digital tools have appeared, there are still systematic problems: for example, the image modeling tool only supports static joint space measurement and lacks dynamic occlusion simulation capability, and cannot predict the therapeutic jaw position; the electronic facebow system can record the jaw movement trajectory, but the data is not linked with the jaw pad design end, resulting in disconnection between treatment planning and production; the 3D printed jaw pad shortens the manufacturing cycle, but the retention force design depends on manual experience, and improper processing of the undercut leads to a dismounting rate of more than 20%.

[0005] In summary, the current jaw pad preparation scheme has many defects: First, there is a precision and efficiency bottleneck: the joint positioning error of the manual jaw position taking method is greatly affected by the doctor's experience, and the digital jaw position taking method cannot generate a high-precision treatment navigation scheme due to the isolation of CBCT, oral scanning, and motion trajectory data from multiple sources; Second, dynamic function evaluation is missing; the traditional jaw frame evaluation only relies on static imaging parameters, ignores the biomechanical effects of opening and closing and lateral movement, and lacks treatment preview technology, making it impossible for doctors to predict the condyle reduction effect after the patient wears the jaw pad; Third, the manufacturing difficulty is high: the retention force design of the jaw position highly depends on the experience of technicians, which is prone to problems such as uneven control of undercut depth and angle during the manufacturing process, and manual or desktop 3D printed jaw pads have large edge thickness fluctuations, which also easily cause local pain and mucosal damage; Fourth, the clinical efficiency is low: the diagnosis, planning, and production links of the current jaw pad manufacturing are separated, resulting in a long average clinical treatment cycle, many patient rechecks, and low treatment efficiency. SUMMARY

[0006] The embodiment of the application provides a generation method of a digital jaw pad based on joint intelligent interaction system navigation.

[0007] In a first aspect, the embodiment of the application provides a generation method of a digital jaw pad based on joint intelligent interaction system navigation, comprising the following steps: S1: acquiring multi-modal data of a patient and inputting the multi-modal data into a multi-modal fusion model to generate a digital dental jaw three-dimensional model and analysis results, wherein the multi-modal data comprises temporomandibular joint CBCT images, oral scan STL data and mandibular motion trajectory data, and the analysis results comprise TMD typing results and abnormal quantification parameters; S2: acquiring individual feature data of the patient, inputting the individual feature data into a standard data adjustment model to output individual quantification parameters of the current patient, wherein the individual feature data comprises age, gender, occlusion type and jaw bone shape; calculating an adaptability value based on the difference between the joint cavity gap of the digital dental jaw three-dimensional model and the normal quantification parameters; and adopting a multi-objective optimization algorithm to iteratively adjust the position of the mandibular bone of the digital dental jaw three-dimensional model to a treatment position with the joint target of "centring the condyle in the joint fossa, uniformizing the joint gap and minimizing muscle stress"; S3: processing the oral scan STL data through an intelligent design engine to obtain a concave region; S4: manufacturing a digital jaw pad based on the position of the mandibular bone in the treatment position and the concave region.

[0008] The main contributions and innovations of the application are as follows: The scheme discloses a generation method of a digital jaw pad based on a joint intelligent interaction system navigation and related devices, and the core is to solve the precision, efficiency and process pain points of traditional jaw pad preparation through a full-process closed-loop design. The method process mainly includes four steps: first, acquiring multi-modal data such as temporomandibular joint CBCT images, oral scan STL data and mandibular motion trajectory data, generating a digital dental jaw three-dimensional model and analysis results such as TMD typing and abnormal quantification parameters through a multi-modal fusion model; second, combining individual features such as patient age and gender, obtaining individual quantification parameters through a standard data adjustment model, and iteratively adjusting the mandibular bone to a treatment position of "centring the condyle, uniformizing the gap and minimizing muscle stress" through a multi-objective optimization algorithm; third, accurately obtaining a tooth concave region through point cloud preprocessing, mesh reconstruction, concave identification and completion through an intelligent design engine; and fourth, manufacturing a high-precision digital jaw pad based on the treatment position and the concave region. The scheme innovatively adopts multi-modal data fusion, AI intelligent design, precise algorithm optimization and other technologies, breaks through the limitations of traditional manual or partial digitalization schemes, and realizes the overall improvement of diagnosis and treatment precision, manufacturing process and clinical efficiency.

[0009] The details of one or more embodiments of the application are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the application will be apparent from the description of illustrative embodiments of the application and from the claims. BRIEF DESCRIPTION OF DRAWINGS

[0010] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings: Figure 1 is a flow chart of a method for generating a digital jaw pad based on joint intelligent interaction system navigation according to an embodiment of the application; Figure 2 is a schematic diagram of a digital dental jaw three-dimensional model generated by an embodiment of the application.

[0011] Figure 3 is a schematic diagram of the position of the mandible in the treatment position generated by an embodiment of the application.

[0012] Figure 4 is a technical logic diagram of a global feature extraction module of the present solution.

[0013] Figure 5 is a schematic diagram of a reverse recessed area.

[0014] Figure 6 is a schematic diagram of a digital jaw pad designed according to an embodiment of the application.

[0015] Figure 7 is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the application. DETAILED DESCRIPTION

[0016] The illustrative examples set forth in the following description have been presented for purposes of illustration. In the following description, for example, numerous specific details are set forth in order to provide a thorough understanding of various embodiments of the present application. However, a person of ordinary skill in the art will realize and appreciate that the various embodiments of the present application can be practiced without resorting to these specific details. In some instances, well-known methods and methods have not been described in detail in order to avoid unnecessarily obscuring aspects of the present application.

[0017] It should be noted that the steps of the methods in other embodiments need not necessarily be performed in the order indicated in this specification. In some other embodiments, the steps included in the methods can be more or fewer than those described in this specification. Furthermore, a single step described in this specification can be split into multiple steps in other embodiments; and multiple steps described in this specification can be combined into a single step in other embodiments.

[0018] Embodiment one As Figure 1 shown, the present scheme provides a method for generating a digital jaw pad based on joint intelligent interaction system navigation, comprising the following steps: S1: acquiring multi-modal data of the patient and inputting the multi-modal data into a multi-modal fusion model to generate a digital dental jaw three-dimensional model and analysis results, wherein the multi-modal data includes temporomandibular joint CBCT images, oral scan STL data, and mandibular motion trajectory data, and the analysis results include TMD typing results and abnormal quantitative parameters; S2: acquiring individual characteristic data of the patient, inputting the individual characteristic data into a standard data adjustment model to output individual quantitative parameters of the current patient, wherein the individual characteristic data includes age, gender, bite type, and jaw bone shape; calculating the fitness value based on the difference between the joint cavity gap of the digital dental jaw three-dimensional model and the normal quantitative parameters, taking the joint goals of “condyle centered in the joint fossa, uniformization of the joint gap, and minimization of muscle stress”, and using a multi-objective optimization algorithm to iteratively adjust the position of the digital dental jaw three-dimensional model to the treatment position; S3: processing the oral scan STL data through an intelligent design engine to obtain the undercut area; S4: making a digital jaw pad based on the position of the mandibular bone in the treatment position and the undercut area.

[0019] Regarding step S1: The present scheme acquires multi-modal data of the patient including temporomandibular joint CBCT images, oral scan STL data, and mandibular motion trajectory data, and fuses the multi-modal data to overcome the problem of independent data sources in traditional schemes, forming a synergy of anatomical structure, geometric shape, and dynamic function, thereby greatly improving the positioning accuracy of the condyle.

[0020] The temporomandibular joint CBCT image is three-dimensional anatomical information collected by cone beam computed tomography technology, and records the hard tissue anatomical information of the patient's temporomandibular joint and jaw bone in the temporomandibular joint CBCT image. Specifically, the temporomandibular joint CBCT image includes the shape and spatial position of the fossa and condyle, the disc gap size, the integrity of the bone structure (such as whether there are changes in bone quality, defects), whether there is effusion in the joint cavity, etc. Static anatomical features provide a high-precision structural basis for subsequent joint positioning and gap measurement.

[0021] The oral scan STL data is three-dimensional geometric data of the tooth and gum surface collected by an oral scanner and stored in an STL (stereolithography) format as a point cloud file. The oral scan STL data records the overall morphology of the dentition, the surface profile of individual teeth, the adjacent relationship between teeth, the distribution and depth of undercut areas on the tooth surface, the characteristics of the dental arch curve, and other details, accurately restores the geometric morphology of the oral hard tissue, and is the core geometric basis for the design and fit optimization of the jaw pad retention structure.

[0022] The mandibular movement trajectory data is dynamic time series data collected by an electronic facebow system, which records the mandibular movement characteristics of the patient during the opening and closing of the mouth, lateral movement, and mastication. The core content includes the size of the opening degree, the smoothness of the movement trajectory, the symmetry of the condylar movement center, the specific angle position corresponding to the joint click, the trajectory deviation or tremor in the movement cycle, and other dynamic parameters, which fully presents the functional state of the mandibular movement and provides data support for dynamic function evaluation and treatment simulation.

[0023] Further, the multi-modal fusion model of the present scheme includes a multi-modal data encoding layer, a cross-modal feature alignment module, a model output module, and a TMD typing decoder connected in sequence, wherein the multi-modal data encoding layer is used for independent encoding processing of the multi-modal data to obtain encoding features of different modalities, the cross-modal feature alignment module is used for aligning the encoding features of different modalities to obtain cross-modal alignment features, the model output module is used for decoding processing of the cross-modal alignment features to obtain a digitalized dental and jaw three-dimensional model, and the TMD typing decoder is used for decoding processing of the cross-modal alignment features to obtain an analysis result.

[0024] Specifically, the multi-modal data encoding layer of the present scheme includes an independent 3D CNN network, a PointNet++ network, and a time series network, wherein the temporomandibular joint CBCT image is input into the 3D CNN network for encoding processing to obtain CT image encoding features, the oral scan STL data is input into the PointNet++ network for encoding processing to obtain three-dimensional point cloud features, and the mandibular movement trajectory data is input into the time series network for encoding processing to obtain time series features.

[0025] In some embodiments, the multi-modal data encoding layer of the present scheme segments the temporomandibular joint CBCT image through the 3D CNN network to extract the anatomical joint as the encoding features of the CT image, i.e., the CT image encoding features include the features of the anatomical joint.

[0026] In some embodiments, the multi-modal data encoding layer of the present scheme processes the oral scan STL data through the PointNet++ to obtain the encoding features of the corresponding STL data, wherein the encoding features of the corresponding STL data are three-dimensional point clouds, which can be used to construct a tooth surface point cloud model.

[0027] In some embodiments, the multi-modal data encoding layer of the present scheme processes the mandibular movement trajectory data through a timing network to obtain encoded features of the corresponding movement trajectory data for representing the movement timing.

[0028] Specifically, the timing network is a hybrid architecture composed of a bidirectional long short-term memory network and a Transformer encoding layer, wherein the bidirectional long short-term memory network is responsible for capturing local timing features such as trajectory shift and tremor in the mandibular movement trajectory data, and the output of the bidirectional long short-term memory network is input into the Transformer encoding layer, which uses a self-attention mechanism to analyze the cross-period dependence in long-time movement to obtain encoded features.

[0029] Specifically, the cross-modal feature alignment module of the present scheme is used to align the encoded features of different modalities to obtain aligned features. Further, the cross-modal feature alignment module realizes the semantic association of anatomical structure and dynamic trajectory based on a cross-attention mechanism.

[0030] In some embodiments, the encoded features from the temporomandibular joint CBCT image and the oral scan STL data are used as anatomical structure features, and the encoded features from the mandibular movement trajectory data are used as dynamic trajectory features. The anatomical structure features and the dynamic trajectory features are input into the cross-modal feature alignment module for feature dimension normalization processing and then processed by the cross-attention mechanism to obtain cross-modal aligned features.

[0031] Specifically, the anatomical structure features are used as the query of the cross-attention mechanism, and the dynamic trajectory features are used as the key and value of the cross-attention mechanism to calculate the first cross-attention weight. The dynamic trajectory features are used as the query of the cross-attention mechanism, and the anatomical structure features are used as the key and value of the cross-attention mechanism to calculate the second cross-attention weight. The anatomical structure features are enhanced based on the first cross-attention weight to obtain anatomical structure enhanced features, and the dynamic trajectory features are enhanced based on the second cross-attention weight to obtain dynamic trajectory enhanced features. The anatomical structure enhanced features and the dynamic trajectory enhanced features are added or spliced at the element level to obtain the cross-modal aligned features.

[0032] Further, the model output module of the present scheme is used to decode the cross-modal aligned features to obtain a digital dental and jaw three-dimensional model. The output digital dental and jaw three-dimensional model is shown in Figure 1 .

[0033] The TMD typing decoder of the present scheme is used to decode the cross-modal aligned features to obtain analysis results. Specifically, multiple parallel task learning heads are set in the TMD typing decoder to output analysis results, wherein the analysis results include TMD typing results and abnormal quantitative parameters.

[0034] In some embodiments, the TMD typing decoder comprises a TMD typing task head and an abnormal parameter quantification task head, wherein the TMD typing task head adopts a fully connected layer and a Softmax activation function architecture to map the adapted cross-modal features into a preset typing category space to output a TMD typing result, and the abnormal parameter quantification task head also adopts a fully connected layer and a Softmax activation function architecture to map the adapted cross-modal features into a preset abnormal parameter space to output an abnormal quantification parameter.

[0035] Further, the TMD typing result comprises 8 main TMD typing categories, and the abnormal quantification parameter comprises an anatomical abnormal parameter, a functional abnormal parameter, and a correlation strength abnormal parameter, wherein the anatomical abnormal parameter comprises a disc displacement distance, an asymmetry degree of joint space, a bone defect area, and the like, the functional abnormal parameter comprises an output opening degree, a trajectory smoothness, a condyle center of motion symmetry, and the like, and the correlation strength parameter comprises a structural abnormality - functional abnormality correlation confidence.

[0036] Regarding the training method of the multi-modal fusion model, in the embodiments of the present scheme, the multi-modal fusion model is trained relying on a dynamic weight adjustment and a federated learning framework, and the specific training process is not described.

[0037] As described above, the multi-modal fusion model provided by the present scheme can deeply mine image information: automatically identify key anatomical abnormal features such as disc position, bone changes, and effusion by using a 3D CNN, surpassing the limitations of naked eye observation, and effectively capture subtle functional abnormal features such as mandibular movement trajectory deviation, tremor, and joint sound patterns through a time series network, and can effectively fuse heterogeneous data: realize semantic association of multi-modal information through a cross-modal feature alignment module, and serve the final accurate typing decision together, which significantly improves the ability of the model to handle complex TMD cases, and is more advantageous for the diagnosis of mixed TMD.

[0038] Regarding step S2: The standard data adjustment model constructed in the present scheme can consider the normal value range or proportional relationship of the joint space of individual differences to adjust the individual quantification parameter.

[0039] In some embodiments, the individual quantification parameter focuses on key diagnosis and treatment indicators of the temporomandibular joint, and the individual quantification parameter comprises a joint space parameter, a symmetry parameter, a condyle centering parameter, and a dynamic function parameter, wherein the joint space parameter records the normal value range and proportional relationship of the anterior joint space, the superior joint space, and the posterior joint space; the condyle centering parameter records the normal interval of the linear ratio; the symmetry parameter records the maximum allowable difference of the bilateral joint spaces; and the dynamic function parameter records the normal range of the opening degree, the motion trajectory smoothness threshold, and the like.

[0040] In some embodiments, individual characteristic data of the patient is collected, the age and gender in the individual characteristic data are obtained from the patient's interview or medical record data, and the bite type and jaw shape are obtained from the TMD typing result, i.e., the bite type and jaw shape of the patient can be known through the TMD typing result of step S1.

[0041] The method for constructing the standard data adjustment model is as follows: Normal quantitative parameters of a healthy population are collected, and a healthy population database is constructed based on the normal quantitative parameters, wherein the normal quantitative parameters also include joint space parameters, symmetry parameters, condyle centering parameters, and dynamic function parameters. The normal quantitative parameters of the healthy population database are adjusted based on the individual characteristic data to obtain individual quantitative parameters.

[0042] Specifically, the normal quantitative parameters are stored in layers according to the individual characteristic dimensions of the individual characteristic data, the normal quantitative parameters corresponding to each individual characteristic are matched based on the obtained individual characteristic data, and the individual quantitative parameters are obtained by adjusting the normal quantitative parameters based on the individual characteristics and using a multi-factor weighting algorithm.

[0043] In some embodiments, the multi-factor weighting algorithm includes a preset characteristic weight distribution rule and a characteristic adjustment coefficient, wherein the characteristic weight distribution rule records the influence weights of the bite type, the jaw shape, the age, and the gender, and the characteristic adjustment coefficient records adjustment coefficients corresponding to different bite types, jaw shapes, ages, and genders. Correspondingly, the adjustment coefficients and the influence weights corresponding to the individual characteristics are retrieved from the multi-factor weighting algorithm according to the individual characteristics, the normal quantitative parameters corresponding to the current individual characteristics are weighted by the adjustment coefficients and the influence weights, and the individual quantitative parameters are obtained by summarizing.

[0044] For example, based on the healthy population database, a "normal quantitative parameter system considering individual differences" is constructed, which mainly includes joint space ratios (anterior space: upper space: posterior space ≈ 1.00:1.07:1.86), condyle centering quantitative indicators (linear ratio LR=(P-A) / (P+A)*100, normal range is -12 to +12), bilateral symmetry threshold (gap difference ≤0.5mm is considered normal), and dynamic function parameters (such as normal range of mouth opening). In actual application, the system first calls the multi-factor weighting algorithm to dynamically adjust the normal range threshold based on the individual data of the patient, for example, for an Angle II malocclusion patient, the algorithm sets the bite type as the core influencing factor (weight 0.3) and automatically relaxes the upper limit of the posterior space normal value; for patients with joint fossa depth >8mm, the LR value normal range is fine-tuned to -13 to +13 through the characteristic adjustment coefficient, and finally the individual quantitative parameters suitable for the patient are generated.

[0045] Subsequently, the system compares the automatically measured actual joint space value, LR value, and bilateral gap difference in the patient's digital dental jaw three-dimensional model with the above-mentioned personalized normal quantitative parameters in multiple dimensions. If it is detected that the patient's posterior joint space is significantly widened (more than 20% above the upper limit of the personalized normal range) and the LR value is far above +12, it is identified as an abnormal pattern with clinical significance, suggesting a high possibility of anterior disc displacement. At the same time, the system cross- verifies the dynamic characteristics of joint clicking and trajectory deviation in the mandibular movement trajectory data to exclude isolated data errors, and finally outputs the accurate TMD classification result and abnormal data report through the TMD classification decoder, providing core diagnostic basis for subsequent condylar intelligent repositioning and jaw pad intelligent design.

[0046] Further, unlike the traditional method of subjectively adjusting the position of the mandible by the operator, the present scheme iteratively adjusts the position of the mandible of the digital dental jaw three-dimensional model to restore the mandible to the normal position through a multi-objective optimization algorithm.

[0047] Specifically, the present scheme calculates the fitness value based on the difference between the joint cavity gap of the digital dental jaw three-dimensional model and the normal quantitative parameter, and iteratively adjusts the position of the mandible of the digital dental jaw three-dimensional model to the treatment position using a multi-objective optimization algorithm with the joint objectives of "centering the condyle in the joint fossa, uniformizing the joint space, and minimizing muscle stress".

[0048] The implementation of the multi-objective optimization algorithm is as follows: Randomly generate or initialize a set of potential mandibular positions based on clinical prior knowledge as a population, where each mandibular position is composed of parameters representing translation and rotation, and set the joint objectives of "centering the condyle in the joint fossa, uniformizing the joint space, and minimizing muscle stress" as the joint objective function, and perform iteration of the population; wherein in each iteration, the fitness value corresponding to each mandibular position in the current population is calculated, the superior individuals are selected according to the fitness value, and the superior individuals are recombined and randomly mutated to generate a new offspring, and the new offspring is continued to be iterated until the optimal solution is obtained, and the position of the mandible of the optimal solution is taken as the treatment position.

[0049] Regarding crossover recombination: randomly select two high-quality individuals to exchange part of the position parameters to generate a new candidate solution for the next generation iteration; regarding random mutation: randomly fine-tune some position parameters of an individual with a small probability to increase the diversity of the population and avoid the algorithm from falling into local optimum too early.

[0050] In addition, during the iteration process of the population, a new set of offspring is generated, and a real-time occlusion analysis engine is called to predict the tooth contact point through geometric collision detection. Once unstable early contact or interferences, the solution will be directly excluded or given a very low fitness. The advantage is that the adjustment after prediction by the real-time occlusion analysis engine will be The contact points and the force flow direction, dynamically exclude physiologically unfeasible interference solutions, and the final output is a data-driven, quantifiable "treatment position" recommendation, rather than a trial-and-error adjustment relying on the experience of the physician.

[0051] In some embodiments, when a preset maximum number of iterations is reached or the solution set satisfies the convergence condition, the algorithm terminates, and finally selects the mandibular position that is the most comprehensive in each index from the optimal solution set as the data-driven, quantifiable "treatment position".

[0052] Further, set the quantification target function of "condyle centered in the glenoid fossa", the quantification target function of "uniformization of joint space", and the quantification target function of "minimization of muscle force", and weight each quantification target function to obtain a joint target function, wherein the quantification target function of "condyle centered in the glenoid fossa" takes the linear ratio as the core quantification index, the goal is to make the LR value in the centered interval of -12 to +12, ensuring the spatial position of the condyle in the glenoid fossa is symmetrical and without offset; the quantification target function of "uniformization of joint space" takes the proportional relationship of "anterior space: upper space: posterior space ≈ 1.00:1.07:1.86" as the benchmark, while controlling the difference value of bilateral joint space ≤0.5mm, to avoid local joint overload; the quantification target function of "minimization of muscle force" simulates the muscle tension and occlusal force when the digital dental three-dimensional model moves, the goal is to make the muscle force value lower than 1.2 times the average value of healthy people, to ensure that the treatment position meets the physiological mechanics law.

[0053] Further, calculate the joint cavity space of the digital dental three-dimensional model, which includes the anterior space, the upper space, and the posterior space, wherein the anterior space is the minimum distance between the anterior extreme point of the condyle and the anterior wall of the glenoid fossa, the upper space is the minimum distance between the top point of the condyle and the top of the glenoid fossa, and the posterior space is the minimum distance between the posterior extreme point of the condyle and the posterior wall of the glenoid fossa.

[0054] In some embodiments, the size and spatial distribution of each space can be visually displayed in the digital dental three-dimensional model in the form of color mapping, distance marking lines, etc. A space distribution chart / table can be generated.

[0055] Further, based on the anterior space, the upper space, and the posterior space, respectively calculate the absolute value of the difference between the current space value and the personalized normal parameter value, and then weight and sum the absolute values of the differences to obtain the joint space difference value; calculate the absolute deviation of the current LR value from the midpoint of the centered interval as the condyle centering difference value; calculate the difference between the current muscle force value and the average value of healthy people to obtain the muscle force difference value; and weight the joint space difference value, the condyle centering difference value, and the muscle force difference value to obtain the fitness value of the current mandibular position.

[0056] It should be noted that if the LR value exceeds the interval of-12 to +12, an additional penalty coefficient is superimposed when calculating the neutral difference of the condyle (e.g. the penalty coefficient increases by 0.1 for each unit of deviation); if the difference between the current muscle force value and the mean value of the healthy population is greater than 1.2, the double difference calculation is triggered.

[0057] Regarding the "treatment position" output in step S2, the "treatment position" records the 6-degree-of-freedom position parameters of the mandible, which can be visually displayed on the digital dental arch three-dimensional model. This 6-degree-of-freedom position parameter and the digital dental arch three-dimensional model are direct digital basis for making therapeutic bite plates (precise adjustment of bite height and jaw position) or guiding condyle reduction targets in orthognathic / joint surgery. Of course, the operator can comprehensively evaluate the "treatment position" generated in step S2 to evaluate the improvement of the joint space, the occlusal contact situation (if simulated), and the clinical feasibility and make adjustments. The "treatment position" displayed on the digital dental arch three-dimensional model is as shown in Figure 3 .

[0058] Regarding step S3: The present scheme processes the oral scan STL data through an intelligent design engine, automatically identifies the undercut area of the tooth surface and accurately calculates the key parameters through a three-step process of "point cloud preprocessing → mesh reconstruction → undercut identification and quantification", without human intervention throughout.

[0059] Specifically, step S3 further comprises the steps of: standardizing the oral scan STL data to obtain standard point cloud data, reconstructing the tooth surface triangular mesh based on the standard point cloud data to obtain a complete mesh model, and identifying the undercut area on the complete mesh model and calculating the key parameters.

[0060] In the step of "standardizing the oral scan STL data to obtain standard point cloud data", the point cloud center of the oral scan STL data is moved to the origin of the coordinate system by centroid translation and unit sphere normalization, then the PCA principal axis alignment is used to automatically calibrate the dental arch symmetry axis direction, and finally the standard point cloud data is obtained by random downsampling to retain key geometric features.

[0061] Specifically, the input to the intelligent design engine of the present scheme is the untreated oral scan STL data, which contains noise, uneven density, and spatial position and scale differences caused by scanning posture and distance. Therefore, the intelligent design engine of the present scheme first uses standardization processing to obtain aligned and normalized standard point cloud data, providing standardized input data for subsequent automatic identification of undercut areas, mesh generation, and point cloud completion.

[0062] Further, the point cloud center of the mouth scan STL data is moved to the origin of the coordinate system to facilitate subsequent symmetric or asymmetric undercut analysis centered on the tooth area. Different sizes of dental models are uniformly scaled to a standard scale to eliminate the scale effects caused by different scanning distances or patient arch sizes, providing a uniform basis for subsequent algorithm parameters (such as neighborhood search radius, undercut determination threshold). Finally, the principal direction of the point cloud (e.g., the symmetric axis direction of the dental arch) is automatically calculated by the PCA principal axis analysis method, and the point cloud is rotated to the standard direction to automatically calibrate the symmetric axis direction of the dental arch. This ensures that subsequent undercut detection, mesh generation, and other operations are based on consistent poses, improving algorithm robustness and result comparability, which is crucial for subsequent precise positioning of adjacent surfaces and calculation of undercut regions.

[0063] The implementation of the "standardizing the mouth scan STL data to obtain standard point cloud data" is as follows: calculate the geometric center of the mouth scan STL data as the point cloud center, and take the point cloud center as the origin to calculate the distance of the farthest point from the origin. Based on the distance, the entire point cloud is scaled to a unit sphere for unit sphere normalization. The principal direction matrix of the point cloud is calculated by principal component analysis, and the point cloud is projected onto the direction. A certain number of points are randomly selected from the aligned point cloud as standard point cloud data.

[0064] In the "reconstructing a triangular mesh of the tooth surface based on the standard point cloud data to obtain a complete mesh model" step, the spatial scale features of the point cloud are calculated based on the standard point cloud data, the alpha value is adaptively calculated based on the spatial scale features, and a complete mesh model is generated through Delaunay triangulation and alpha-envelope operation.

[0065] This scheme generates a smooth triangular mesh surface outside the point cloud through the alpha-envelope (Alpha Wrap) algorithm to approximate the geometric shape of the original point cloud. Compared with the traditional alpha-shape algorithm, Alpha Wrap uses two parameters to control the detail preservation and closure of the reconstruction, and can obtain more stable reconstruction results on point clouds with high noise or uneven point density.

[0066] In some embodiments, CGAL::IO::read_points() is called to read standard point cloud data from a file and verify the validity and non-emptiness of the read results. If the point cloud is empty, the process is terminated directly.

[0067] In some embodiments, the spatial scale features include a diagonal length, an average point distance, and a feature distance. Specifically, a spatial diagonal length of a bounding box of the point cloud is calculated as the diagonal length, a mean of distances between all adjacent points is calculated to obtain the average point distance, and a distance between key features of the teeth is extracted as the feature distance. Specifically, the spatial scale features are obtained by calculating the distances between adjacent points and the distances between feature points one by one through the traversal of point cloud coordinates using the Euclidean distance formula, and then obtaining the spatial scale features by averaging.

[0068] Further, in the Alpha Wrap algorithm of CGAL, the adaptive calculation of the alpha parameter mainly aims to enable the algorithm to intelligently determine a suitable alpha value according to the characteristics of the input point cloud data, so as to achieve a good balance between preserving geometric details and generating a smooth mesh. The core idea is to associate the alpha value with the scale features of the point cloud itself.

[0069] In some embodiments, the alpha value is obtained by weighting the diagonal length, the average point distance, and the feature distance, and the alpha value needs to be constrained in a preset interval to avoid mesh fragmentation caused by too small alpha value and loss of details caused by too large alpha value. After obtaining the alpha value, a complete mesh model is generated through Delaunay triangulation and alpha-envelope operation. Specifically, the basic geometric topology of the point cloud is constructed through Delaunay triangulation, and then the envelope conforming to the alpha radius and offset limit is calculated through the alpha value, and the outer surface of the envelope is extracted as the complete mesh model.

[0070] In some embodiments, the mesh closure of the complete mesh model is checked, and the closed complete mesh model is saved as a ply file.

[0071] The scheme realizes the logic of undercut in CAD software, and the geometric essence is to accurately create a circular arc or a curved surface tangent to the adjacent edge line in a two-dimensional or three-dimensional model through a core algorithm. In specific implementation, the system automatically performs a Boolean difference set operation according to the edge selected by the user and the specified fillet radius, that is, simulates the "cutting" of the material along the edge with a spherical cutter to form a uniform recess. This process needs to follow a strict modeling sequence to ensure successful generation. With the introduction of AI technology, the undercut logic is developing from manual operation to intelligence. For example, through technologies such as CAD-Tokenizer, the invalid design rate is reduced, and the system is given the ability of hierarchical understanding and context perception, so that it can automatically judge and generate a reasonable undercut structure that meets the design intent and process requirements according to the function, aesthetics, and manufacturing constraints (such as the ejection angle) of the part. The intelligent design engine of the present scheme identifies the suspected undercut area based on the undercut recognition algorithm and optimizes it to obtain an accurate undercut area.

[0072] Further, in the step of "identifying undercut regions on the complete mesh model and calculating key parameters", a suspected undercut region is identified in the complete mesh model by an undercut identification algorithm, a subset of vertices of the suspected undercut region is extracted to form a partial point cloud region, and the partial point cloud data is input into the point cloud completion model to complete the undercut region point cloud. The undercut region point cloud is mapped back to the complete mesh model to obtain the undercut region and the key parameters.

[0073] It should be noted that although the traditional undercut identification algorithm can identify the suspected undercut region, there may be sparseness / loss due to scanning occlusion, and therefore the present scheme further completes the undercut region point cloud by the point cloud completion model. That is, the partial point cloud region is a key region circled for analyzing the undercut or completing the damage.

[0074] In some embodiments, the point cloud completion model includes a parallel global feature extraction module, a local feature encoding module, and a multi-level refinement module connecting the global feature extraction module and the local feature encoding module, wherein the global feature extraction module includes a parallel 3D point cloud feature encoder and a 2D image feature encoder, a feature fusion module connected to the 3D point cloud feature encoder and the 2D image feature encoder, and a decoder. The point cloud provided by the present scheme uses the point cloud completion model to complete the sparseness / loss details of the partial point cloud, restores the complete geometric shape (such as the deepest point coordinates and the concave edge contour) of the undercut region, and solves the limitations of clinical scanning data.

[0075] Specifically, as shown in FIG. 2, Figure 4 Figure 4 is the technical logic of the global feature extraction module of the present scheme. At this time, the partial point cloud region is rendered to obtain a depth view, and the partial point cloud region and the corresponding depth view are input into the global feature extraction module. The point cloud data of the partial point cloud region is input into the 3D point cloud feature encoder to extract 3D global features, the depth view is input into the 2D image feature encoder to obtain 2D multi-view features, the 3D global features and the 2D multi-view features are input into the cross-modal fusion module for cross-modal fusion to obtain global features, and the global features are input into the decoder to obtain a rough point cloud. The global feature extraction module of the present scheme extracts global features from the partial point cloud and its depth view, which can be understood as allowing the global feature extraction module to "observe" and understand the object from different dimensions at the same time to obtain more comprehensive overall shape information.

[0076] ​Further, the point cloud data of the partial point cloud region is input into the local feature encoding module to extract local geometric features, which are used to capture detailed structural information, i.e., the role of the local feature encoding module is to convert the geometric information (curvature, normal change, etc.) of the local region around the key point into a numerical vector (i.e., a feature descriptor, so as to be able to distinguish different local shapes, and the extraction manner can be mainly divided into two categories of traditional descriptors based on manual rules and deep learning descriptors based on data driving.

[0077] Further, the multi-level refinement module includes a fusion module and a refinement module, and the rough point cloud and the point cloud data of the original partial point cloud region are input into the fusion module to obtain fusion features, and the fusion features and the local geometric features are input into the refinement module to obtain the undercut region point cloud.

[0078] Further, the fusion module merges the rough point cloud and the point cloud data of the original partial point cloud region and selects feature points as fusion features through farthest point sampling. Farthest point sampling (FPS) is a downsampling algorithm in point cloud processing, and the core idea is to iteratively select the point farthest from the current selected point set, the purpose is to cover the entire spatial structure of the original point cloud as evenly as possible with fewer points, so as to retain key geometric features. After the fusion module of the present scheme merges the initially generated rough point cloud and the point cloud data of the original partial point cloud region, the data amount may still be large and the distribution may not be uniform, so through farthest point sampling, a representative point set with less quantity but more uniform spatial distribution is selected as the fusion feature, so that in the subsequent refinement stage, the details can be more concentrated and the shape can be more fine-tuned on these points with good spatial representation at a lower computational cost, while avoiding model bias caused by uneven point distribution, so as to more effectively capture and reconstruct the overall geometric shape of the object.

[0079] Further, the refinement module includes two SDG sub-modules, and the SDG sub-module is a neural network component for optimizing and generating more fine three-dimensional point clouds by fusing global shape features and local geometric details, and the fusion features and the local geometric features are sequentially input into the SDG sub-module of the refinement module to obtain the undercut region point cloud.

[0080] The training method of the point cloud completion model is not repeated, and the loss function of the point cloud completion model is as follows:

[0081]

[0082]

[0083]

[0084] ; Where P and Q represent any two sets of 3D point clouds that need to be compared. Each point cloud is a set of a large number of 3D coordinate points (x, y, z), representing a 3D object. This indicates that the point cloud P contains N three-dimensional points, each denoted as p. i .

[0085] Let Q be a point cloud containing M three-dimensional points, each denoted as q. j .

[0086] The chamfer distance is an indicator that measures the overall shape difference between two point clouds P and Q. It is calculated in a "two-way" manner: for each point in P, find the nearest point in Q, calculate the square of the distance, and then average it over all points. Similarly, for each point in Q, find the nearest point in P, calculate the square of the distance, and then average it over all points. Finally, add the two averages together. The smaller this value is, the closer the shapes of the two point clouds are.

[0087] P0 corresponds to the generated coarse point cloud; P1 and P2 correspond to the medium-precision point cloud and high-precision point cloud output by the two SDG sub-modules respectively. g t Point cloud representing ground truth; A mathematical transformation (hyperbolic sine function, arcosh) is performed on the basic chamfer distance. This is typically done to better optimize the model in hyperbolic geometric space, making the loss function landscape smoother and easier to train.

[0088] like Figure 5 As shown, Figure 5 This is a schematic diagram of the undercut area. After obtaining the mandibular position of the undercut area and treatment site, this method can generate a digital jaw pad that conforms to both anatomical structure and biomechanical principles. The generated digital jaw pad is shown below. Figure 6 As shown, this scheme can also support occlusal simulation and analysis on the set jaw position relationship. Its design helps to achieve free centralization of the mandible when the cusps intercuspate, as well as physiologically functional protrusion and lateral movement guidance (such as canine guidance mechanism).

[0089] Based on this approach and traditional technology, our team designed corresponding digital jaw pads. The differences in treatment reliability between the digital jaw pads designed using this approach and the jaw pads designed using traditional approaches are shown in Table 1 below. Table 1. Treatment reliability difference table of digital jaw pad of different schemes .

[0090] Moreover, the scheme can greatly improve the positioning accuracy of the condylar process. The scheme integrates 0.25mm-thick CBCT and 20μm CT data and achieves condylar process positioning accuracy ≤0.1mm, condylar process reduction coincidence ≥90%, and recurrence rate reduced to <8%. Using the scheme, the mandibular movement trajectory can be predicted in milliseconds, and the misdiagnosis rate is reduced to <5%. The digital jaw pad designed by the scheme can cover >95% of complex jaw bone shapes, and the clinical shedding rate is <5%.

[0091] Embodiment two The embodiment also provides an electronic device, which refers to Figure 7 , comprising a memory 404 and a processor 402, the memory 404 stores a computer program, and the processor 402 is configured to run the computer program to execute the steps in the method embodiments of any one of the above-mentioned generation methods of the digital jaw pad guided by the intelligent joint interactive system.

[0092] Specifically, the processor 402 can include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The memory 404 can be used to store or buffer various data files required for processing and / or communication, and the computer program instructions executed by the processor 402.

[0093] The processor 402 reads and executes the computer program instructions stored in the memory 404 to implement any one of the above-mentioned generation methods of the digital jaw pad guided by the intelligent joint interactive system.

[0094] Optionally, the electronic device can further include a transmission device 406 and an input / output device 408, wherein the transmission device 406 is connected with the processor 402, and the input / output device 408 is connected with the processor 402.

[0095] The transmission device 406 can be used to receive or send data via a network. The above-mentioned network examples can include wired or wireless networks provided by a communication provider of the electronic device. In one example, the transmission device includes a network adapter (NIC) that can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 406 can be a radio frequency (RF) module for communicating with the Internet in a wireless manner.

[0096] The input and output device 408 is used for inputting or outputting information. In the present embodiment, the input information can be multi-modal data, etc., and the output information can be a digital jaw pad, etc.

[0097] Optionally, in the present embodiment, the above-mentioned processor 402 can be configured to execute the following steps by means of a computer program: S1: acquiring multi-modal data of a patient and inputting the multi-modal data into a multi-modal fusion model to generate a digital dental jaw three-dimensional model and analysis results, wherein the multi-modal data includes temporomandibular joint CBCT images, oral scan STL data, and mandibular motion trajectory data, and the analysis results include TMD typing results and abnormal quantification parameters; S2: acquiring individual characteristic data of the patient, inputting the individual characteristic data into a standard data adjustment model to output individual quantification parameters of the current patient, wherein the individual characteristic data includes age, gender, bite type, and jaw bone shape; calculating a fitness value based on a difference between joint cavity clearance of the digital dental jaw three-dimensional model and normal quantification parameters; taking the joint goals of “centering the condyle in the joint fossa, uniformizing the joint clearance, and minimizing muscle stress” as the joint goals, and iteratively adjusting the position of the mandible of the digital dental jaw three-dimensional model to the treatment position by using a multi-objective optimization algorithm; S3: processing the oral scan STL data by an intelligent design engine to obtain a undercut area; S4: manufacturing a digital jaw pad based on the position of the mandible in the treatment position and the undercut area.

[0098] It should be noted that the specific examples in the present embodiment can refer to the examples described in the above-mentioned embodiments and optional implementation manners, which will not be described herein again.

[0099] In general, the various embodiments can be implemented in hardware or special purpose circuits, software, logic or any combination thereof. Some aspects of the application can be implemented in hardware, while other aspects can be implemented in firmware or software which can be executed by a controller, microprocessor or other computing device, but the application is not limited thereto. While various aspects of the application can be illustrated and described as block diagrams, flow charts, or using some other pictorial representation, it is well understood that these blocks, apparatus, systems, techniques or methods described herein can be implemented in, as non-limiting examples, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controler or other computing devices, or some combination thereof.

[0100] Embodiments of the application can be implemented by computer software executable by a data processor of the mobile device such as in the processor entity, or by hardware, or by a combination of software and hardware. Computer software or program, also called program product, including software routines, applets and / or macros, can be stored in any apparatus-readable data storage medium and they comprise program instructions to perform particular tasks. The program product can include one or more computer-executable components such as by the program instructions of one or more applications, or portions thereof. The one or more computer-executable components can be one or more of initial applications, utility programs, background services, system services, program libraries, and so on.The one or more computer-executable components can be stored on the physical media as physical media computer instructions, or loaded into memory from another computer readable medium. The one or more computer-executable components can include, but are not limited to, software for implementation of the embodiments. The program instructions can be implemented in a high level procedural or object oriented programming language, or in assembly or machine language. The software can be

[0101] It should be apparent to those skilled in the art that the above-described embodiments can be combined in any manner, and for brevity, not all possible combinations are described above, however, any combination of the above-described features are considered to be within the scope of the present disclosure.

[0102] The above embodiments are only some of the several embodiments of the present application, and the description is relatively specific and detailed, but it should not be understood as limiting the scope of the present application. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of the present application. Therefore, the scope of the present application should be subject to the appended claims.

Claims

1. A method for generating a digital jaw pad based on a joint intelligent interactive system navigation, characterized in that, Includes the following steps: S1: Acquire the patient's multimodal data and input the multimodal data into the multimodal fusion model to generate a digital three-dimensional model of the jaw and analysis results. The multimodal data includes temporomandibular joint CBCT images, intraoral scan STL data and mandibular movement trajectory data. The analysis results include TMD classification results and abnormal quantification parameters. S2: Obtain the patient's individual characteristic data, input the individual characteristic data into the standard data adjustment model, and output the current patient's individual quantitative parameters. The individual characteristic data includes age, gender, occlusion type, and jawbone morphology. The fitness value is calculated by the difference between the joint space and normal quantitative parameters of the digital three-dimensional model of the jaw. With the joint goal of "the condyle is centered in the glenoid fossa, the joint space is homogenized, and the muscle force is minimized", the position of the mandible in the digital three-dimensional model of the jaw is iteratively adjusted to the treatment position using a multi-objective optimization algorithm. S3: Obtain the concave region by processing the STL data from the inlet scan using the intelligent design engine; S4: Create a digital jaw pad based on the position of the mandible and the undercut area of ​​the treatment site.

2. The method for generating a digital jaw pad based on a joint intelligent interactive system navigation according to claim 1, characterized in that, The multimodal fusion model includes a multimodal data encoding layer, a cross-modal feature alignment module, a model output module, and a TMD classification decoder connected in sequence. The multimodal data encoding layer is used to independently encode multimodal data to obtain encoded features of different modalities. The cross-modal feature alignment module is used to align the encoded features of different modalities to obtain cross-modal aligned features. The model output module is used to decode the cross-modal aligned features to obtain a digital three-dimensional model of the dentition. The TMD classification decoder is used to decode the cross-modal aligned features to obtain the analysis results.

3. The method for generating a digital jaw pad based on a joint intelligent interactive system navigation according to claim 2, characterized in that, The multimodal data encoding layer includes independent 3D CNN networks, PointNet++ networks, and temporal networks. The temporal network is a hybrid architecture consisting of a bidirectional long short-term memory network and a Transformer encoding layer. The cross-modal feature alignment module uses encoded features from temporomandibular joint CBCT images and intraoral scan STL data as anatomical structural features and encoded features from mandibular movement trajectory data as dynamic trajectory features. The anatomical structural features and dynamic trajectory features are input into the cross-modal feature alignment module for feature dimension normalization and then processed by a cross-attention mechanism to obtain cross-modal alignment features.

4. The method for generating a digital jaw pad based on a joint intelligent interactive system navigation according to claim 1, characterized in that, Normal quantitative parameters of healthy individuals are collected, a database of healthy individuals is constructed based on the normal quantitative parameters, and individual quantitative parameters are obtained by adjusting the normal quantitative parameters of the healthy individuals database based on individual characteristic data. The normal quantitative parameters and individual quantitative parameters include joint space parameters, symmetry parameters, condylar centering parameters, and dynamic functional parameters.

5. The method for generating a digital jaw pad based on a joint intelligent interactive system navigation according to claim 1, characterized in that, A population of potential mandibular positions is randomly generated or initialized based on prior clinical knowledge. Each mandibular position consists of parameters representing translation and rotation. The joint objective function is set as "central condyle position within the glenoid fossa, homogenization of the joint space, and minimization of muscle stress". The population is iterated. In each iteration, the fitness value corresponding to each mandibular position in the current population is calculated. Individuals with excellent performance are selected based on their fitness values. These individuals are then cross-recombined and randomly mutated to generate new offspring. The new offspring are iterated until the optimal solution is obtained. The mandibular position of the optimal solution is taken as the treatment position.

6. The method for generating a digital jaw pad based on a joint intelligent interactive system navigation according to claim 1, characterized in that, The joint space of the digital three-dimensional model of the jaw is calculated. The joint space includes the anterior space, the superior space, and the posterior space. The anterior space is the minimum distance between the anterior pole of the condyle and the anterior wall of the glenoid fossa. The superior space is the minimum distance between the apex of the condyle and the top of the glenoid fossa. The posterior space is the minimum distance between the posterior pole of the condyle and the posterior wall of the glenoid fossa. Based on the anterior space, the superior space, and the posterior space, the absolute value of the difference between the current space value and the personalized normal parameter value is calculated. Then, the weighted sum of the absolute values ​​of each difference is used to obtain the joint space difference. The absolute deviation between the current LR value and the midpoint of the central interval is calculated as the condyle centrality difference; the difference between the current muscle force value and the mean value of the healthy population is calculated as the muscle force difference. The fitness value of the current mandibular position is obtained by weighting the joint space difference, condylar centrality difference, and muscle force difference.

7. The method for generating a digital jaw pad based on a joint intelligent interactive system navigation according to claim 1, characterized in that, Standardized STL data from oral scanning is processed to obtain standard point cloud data. Based on the standard point cloud data, a triangular mesh of the tooth surface is reconstructed to obtain a complete mesh model. Undercut regions are identified on the complete mesh model and key parameters are calculated.

8. The method for generating a digital jaw pad based on a joint intelligent interactive system navigation according to claim 7, characterized in that, In the complete mesh model, a concave region identification algorithm is used to identify suspected concave regions. A subset of vertices in the suspected concave regions is extracted to form a partial point cloud region. The partial point cloud data is input into the point cloud completion model to complete the concave region point cloud. The concave region point cloud is then mapped back to the complete mesh model to obtain the concave region and key parameters.

9. The method for generating a digital jaw pad based on a joint intelligent interactive system navigation according to claim 8, characterized in that, The point cloud completion model includes a parallel global feature extraction module, a local feature encoding module, and a multi-level refinement module connecting the global feature extraction module and the local feature encoding module. The global feature extraction module includes a parallel 3D point cloud feature encoder and a 2D image feature encoder, a feature fusion module connecting the 3D point cloud feature encoder and the 2D image feature encoder, and a decoder. The multi-level refinement module includes a fusion module and a refinement module.

10. The method for generating a digital jaw pad based on a joint intelligent interactive system navigation according to claim 9, characterized in that, A partial point cloud region is rendered from multiple perspectives to obtain a depth view. The partial point cloud region and the corresponding depth view are input into the global feature extraction module. The point cloud data of the partial point cloud region is input into the 3D point cloud feature encoder to extract 3D global features. The depth view is input into the 2D image feature encoder to obtain 2D multi-view features. The 3D global features and 2D multi-view features are input into the cross-modal fusion module for cross-modal fusion to obtain global features. The global features are input into the decoder to obtain a coarse point cloud. The point cloud data of the partial point cloud region is input into the local feature encoding module to extract local geometric features. The coarse point cloud and the point cloud data of the original partial point cloud region are input into the fusion module to obtain fused features. The fused features and local geometric features are input into the refinement module to refine and obtain the inverted concave region point cloud.

Images