A three-dimensional model regulation method and system for orthodontics

Abstract

The present invention relates to the technical field of neural network models, and specifically relates to a three-dimensional model regulation method and system for tooth correction; the method includes: within a fixed sampling time period, obtaining the saliva interference rate, overexposure degree, and effective pixel amount of the patient's oral cavity at each sampling moment to obtain the modeling stability; clustering according to the modeling stabilities corresponding to multiple sampling time periods to obtain at least two clusters; obtaining the relevance index of each cluster and the membership degree of each sampling time period within the cluster to which it belongs, so as to obtain the modeling reliability of the corresponding sampling time period; obtaining the reconstruction effectiveness, constructing a binary tuple in combination with the modeling reliability, and using the trained neural network to obtain a predicted binary tuple, and regulating the construction of the three-dimensional model based on the predicted binary tuple; improving the accuracy of analysis during the modeling process and ensuring the effect of model regulation.

Description

Technical Field

[0001] The present invention relates to the technical field of neural network models, and particularly to a three-dimensional model regulation method and system for tooth correction. Background Art

[0002] At present, digital dental models are a common and practical technology. Dental clinics scan through oral scanners to form data and upload it to the system. Designers design tooth models on the computer based on the scanned data and then print them through 3D technology. Currently, devices such as mobile phones are equipped with structured light cameras, and dental models can be quickly established through mobile phones. Therefore, the future measurement difficulty will become lower and lower, but the modeling difficulty will gradually increase.

[0003] During the modeling process, the accuracy of using a three-dimensional structured light camera to construct is higher and the speed is faster than that of photos and videos. However, since the camera is actively infrared, it is necessary to regulate inaccurate samples according to the modeling environment of the dental model. Otherwise, due to the influence of the environment, the accuracy of the final three-dimensional model reconstruction will be low and the modeling effect will be poor. Summary of the Invention

[0004] In order to solve the problem of low accuracy and poor effect of the above three-dimensional model reconstruction, the purpose of the present invention is to provide a three-dimensional model regulation method and system for tooth correction, and the specific technical solutions adopted are as follows:

[0005] In a first aspect, an embodiment of the present invention provides a three-dimensional model regulation method for tooth correction, and the method includes the following steps:

[0006] Within a fixed sampling time period, collect the overexposure degree in the patient's oral cavity and obtain the depth map of the patient's oral cavity interior. Based on the depth map, obtain the saliva interference rate and the effective pixel amount; obtain the modeling stability according to the saliva interference rate, the overexposure degree, and the effective pixel amount;

[0007] Cluster based on the modeling stability corresponding to different sampling time periods to obtain at least two clusters; obtain the infrared illuminance mean sequence of all sampling time periods within the cluster, and obtain the correlation index of the corresponding cluster based on the infrared illuminance mean sequence and the modeling stability of each sampling time period within the cluster;

[0008] Obtain the membership degree of each sampling time period within the cluster, and obtain the modeling reliability based on the modeling stability, the membership degree, and the correlation index of the cluster to which the sampling time period belongs for each sampling time period; obtain the reconstruction effectiveness of each sampling time period, and form a binary group with the modeling reliability of the corresponding sampling time period;

[0009] Input the binary tuple of the current sampling period into the trained neural network to obtain the predicted binary tuple for each subsequent sampling period, and regulate the 3D model based on the predicted binary tuple.

[0010] Preferably, the step of obtaining the saliva interference rate and the effective pixel amount based on the depth map includes:

[0011] Obtain the overexposed area in the depth map, and record the pixel points in the depth map except for the overexposed area as the set to be processed;

[0012] Obtain the median of the brightness values of all pixel points in the set to be processed, and sort all pixel points in the set to be processed in descending order based on the brightness value of each pixel point. Select the pixel point set within the first preset ratio after the descending order, and calculate the average value of the brightness values corresponding to all pixel points in the selected pixel point set;

[0013] The absolute value of the difference between the average value and the median is the saliva interference rate;

[0014] Use the internal calculation of the structured light camera that collects the depth map to obtain invalid depth pixel points. The pixel points in the depth map except for the invalid depth pixel points are valid pixel points, and the number of valid pixel points is the effective pixel amount.

[0015] Preferably, the step of obtaining the modeling smoothness according to the saliva interference rate, the overexposure degree, and the effective pixel amount includes:

[0016] Each sampling period includes at least two sampling moments. Obtain the average value and the maximum value of the saliva interference rate at all sampling moments within the sampling period; calculate the ratio of the average value to the maximum value of the saliva interference rate within the sampling period as the first ratio;

[0017] Obtain the variance of the effective pixel amount at all sampling moments within the sampling period, and obtain the maximum value of the overexposure degree at all sampling moments within the sampling period. Construct an exponential function with the negative of the maximum value of the overexposure degree as the power exponent; calculate the sum result of the preset value and the variance of the effective pixel amount, and the ratio of the exponential function to the sum result is the second ratio;

[0018] The product of the first ratio and the second ratio is the modeling smoothness.

[0019] Preferably, the step of clustering based on the modeling smoothness corresponding to different sampling periods to obtain at least two clusters includes:

[0020] Obtain the absolute value of the difference between the modeling smoothness corresponding to different sampling periods as the clustering distance, and divide different sampling periods into at least two clusters based on the clustering distance.

[0021] Preferably, the step of obtaining the infrared illuminance mean value sequence within the cluster and obtaining the relevance index of the corresponding cluster based on the infrared illuminance mean value sequence and the modeling stability of each sampling time period within the cluster includes:

[0022] Obtain the infrared illuminance at each sampling moment in each sampling time period, and calculate the mean value of the infrared illuminance at all sampling moments in each sampling time period; the mean values of the infrared illuminance of all sampling time periods within the cluster constitute the infrared illuminance mean value sequence;

[0023] Perform a descending order arrangement based on the modeling stability of each sampling time period within the cluster to obtain the modeling stability sequence corresponding to the cluster; the elements at the corresponding positions in the modeling stability sequence and the infrared illuminance mean value sequence are the modeling stability and the infrared illuminance mean value of the same sampling time period;

[0024] Obtain the absolute value of the difference between the elements at the corresponding positions in the modeling stability sequence and the infrared illuminance mean value sequence corresponding to each cluster, and sum the absolute values of the differences of all corresponding position elements to obtain the accumulation result;

[0025] Obtain the difference distance between the modeling stability sequence and the infrared illuminance mean value sequence, and obtain the relevance index based on the ratio of the accumulation result to the difference distance.

[0026] Preferably, the step of obtaining the membership degree of each sampling time period within the cluster includes:

[0027] Obtain the average value of the clustering distances between each sampling time period within the cluster and all sampling time periods within the cluster to which it belongs, and normalize the average value of the clustering distances to obtain the membership degree of the sampling time period.

[0028] Preferably, the step of obtaining the modeling reliability degree based on the modeling stability, membership degree of each sampling time period, and the relevance index of the cluster to which the sampling time period belongs includes:

[0029] The calculation formula for the modeling reliability degree is:

[0030]

[0031] Wherein, represents the modeling reliability degree; represents the modeling stability of the sampling time period; represents the difference index of the sampling time period; represents the relevance index of the cluster to which the sampling time period belongs; represents the infrared illuminance mean value of the sampling time period; represents the hyperbolic tangent function; represents the function adjustment parameter;

[0032] The difference index is the product of the membership degree of the corresponding sampling time period and the correlation index of the cluster to which the corresponding sampling time period belongs.

[0033] Preferably, the step of obtaining the reconstruction effectiveness of each sampling time period includes:

[0034] Obtain the point cloud modification area of the corresponding modeling process in the previous sampling time period and the current sampling time period, and normalize the point cloud modification area as the reconstruction effectiveness of the current sampling time period.

[0035] Preferably, the training set of the neural network is a binary group of different patients' oral cavities at different sampling time periods; the weight of the loss function of the neural network is obtained from the membership degree of the corresponding sampling time period.

[0036] In a second aspect, an embodiment of the present invention provides a three-dimensional model regulation system for tooth correction, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above-mentioned three-dimensional model regulation method for tooth correction are implemented.

[0037] The present invention has the following beneficial effects: The embodiments of the present invention are analyzed based on each sampling time period. By obtaining various indicators within the sampling time period, the modeling stability of the sampling time period is calculated. The various indicators of the sampling time period are the saliva interference rate, the effective pixel amount, and the overexposure degree obtained by combining images. The modeling stability of the corresponding sampling time period is calculated using the three indicators, and the combined multi-faceted information results are more reliable; further, at least two clusters are obtained by clustering according to the modeling stability corresponding to different sampling time periods, and the correlation index of the modeling stability of each sampling time period within the cluster and the mean infrared illuminance sequence corresponding to the cluster is obtained, which reflects the current infrared interference situation during the modeling process; then the membership degree of each sampling time period within the cluster is obtained, and the modeling reliability is obtained by combining the modeling stability and the correlation index of the cluster to which it belongs. The modeling reliability is obtained from infrared interference, image features, and the membership degree of the sampling time period itself, ensuring the credibility of the modeling reliability index. A binary group is formed by combining the reconstruction effectiveness corresponding to each sampling time period, and the binary group is used as a whole to input into the neural network to obtain a predicted binary group, which can timely and accurately obtain the modeling situation of the subsequent sampling time period, and based on the predicted binary group, timely adjust the three-dimensional model reconstruction process; it not only improves the accuracy of the modeling reliability analysis during the three-dimensional model modeling process, but also can timely and effectively adjust the three-dimensional model, and the effect of model construction is better. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0039] Figure 1 A flowchart of a three-dimensional model regulation method for tooth correction provided by an embodiment of the present invention. Detailed implementation manners

[0040] To further elaborate on the technical means and effects adopted by the present invention to achieve the intended invention purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation manners, structures, features, and effects of a three-dimensional model regulation method and system for tooth correction proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0041] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs.

[0042] The following specifically describes the specific solutions of a three-dimensional model regulation method and system for tooth correction provided by the present invention with reference to the accompanying drawings.

[0043] Please refer to Figure 1 , which shows a flowchart of a three-dimensional model regulation method for tooth correction provided by an embodiment of the present invention. The method includes the following steps:

[0044] Step S100, within a fixed sampling time period, collect the overexposure degree in the patient's oral cavity and obtain the depth map of the patient's oral cavity interior. Based on the depth map, obtain the saliva interference rate and the effective pixel amount; obtain the modeling smoothness according to the saliva interference rate, the overexposure degree, and the effective pixel amount.

[0045] Specifically, when analyzing and modeling the patient's oral cavity, the presence of saliva in the patient's oral cavity will cause more reflective interference in the oral cavity, which will cause more erroneous information for structured light measurement. Therefore, a method for measuring the non-Lambert saliva interference rate of structured light in the oral cavity is proposed in an embodiment of the present invention. The overexposed area in the depth map is obtained, and the pixels in the depth map except the overexposed area are recorded as a set to be processed; the median of the brightness values ​​of all pixels in the set to be processed is obtained, and all pixels in the set to be processed are arranged in descending order based on the brightness value of each pixel, a set of pixels within a preset ratio after the descending order is selected, and the average value of the corresponding brightness values ​​of all pixels in the selected set of pixels is calculated; the absolute value of the difference between the average value and the median value is the saliva interference rate; the specific measurement and evaluation method of the saliva interference rate is:

[0046] A depth map of the inside of the patient's mouth at each sampling moment is obtained. Excessive saliva in the mouth will cause reflection interference, so when displayed in the image, an overexposed area will appear in the depth map. The overexposed area is the area where the pixel brightness exceeds the range. Generally, the pixel value range of the pixel is 0-255. Taking a conventional mobile phone as an example, it is generally believed that when the value of a pixel in all channels is 255, it is the pixel in the overexposed area. For a single-channel infrared camera, the pixel in the overexposed area is the pixel with a pixel value of 255 in a single channel. The depth map of the inside of the patient's mouth at each sampling moment is analyzed to obtain the overexposed pixels therein, and the connected domain analysis of all the overexposed pixels can obtain the corresponding overexposed area. The method of connected domain analysis is an existing well-known technology and will not be repeated here.

[0047] Furthermore, the overexposed area is expanded at each sampling moment. The expansion method in the embodiment of the present invention adopts a dilation algorithm to obtain the final overexposed connected domain. The dilation algorithm is a commonly used technology in image processing and will not be described in detail. When the depth map is taken, the places with more saliva will be refracted or scattered due to the light spots emitted by the structured light, thereby producing overexposed areas. The overexposed areas will affect the position estimation of the light spots, thereby producing large errors, which will have a great impact on the analysis of the situation in the patient's mouth. Therefore, the overexposed pixels in the overexposed connected domain are discarded, and other normal pixels in the depth map are analyzed, and the remaining pixel set after discarding all the overexposed pixels in the overexposed connected domain is recorded as the set to be processed.

[0048] Set the preset ratio to 30%, select the median of the brightness values of all pixel points in the set to be processed and the top 30% of the pixel points with larger brightness values in the set to be processed. That is, arrange all the pixel points in the set to be processed in descending order according to the brightness values, select the top 30% of the pixel points as top30%, and calculate the average value of the brightness values of the top 30% of the pixel points in the set to be processed; obtain the corresponding saliva interference rate according to the average value of the brightness values of the top 30% of the pixel points in the set to be processed and the median of the brightness values of all pixel points in the set to be processed; the specific calculation of the saliva interference rate is as follows:

[0049]

[0050] Wherein, represents the saliva interference rate; represents the set to be processed; represents the median of all pixel points in the set to be processed; represents the average value of the brightness values of the top 30% of the pixel points in the set to be processed; represents the absolute value calculation.

[0051] The larger the value of

[0052] is, the more uneven the brightness in the set to be processed indicates, and there is a large difference between the average value of the brighter part and the median of the overall brightness. Therefore, the situation of saliva interference is more serious, and the corresponding saliva interference rate is larger.

[0053] Since a depth map of the patient's oral cavity can be obtained at each sampling moment, and each depth map corresponds to a saliva interference rate. In order to establish a temporal analysis, in the embodiments of the present invention, a fixed sampling time period of 10 seconds is set. Within this sampling time period, image acquisition is performed at a sampling frequency of 2 Hz, that is, image acquisition is performed at each sampling moment of 0.5 seconds. Then, within the sampling time period of 10 seconds, 20 depth maps corresponding to the sampling moments can be obtained, and the corresponding saliva interference rates can be obtained based on the 20 depth maps respectively; the settings of the sampling time period and the sampling frequency can be set by the implementer according to different requirements.

[0053] It should be noted that the start time of sampling is the start time of modeling the patient's oral cavity. Store the saliva interference rates at different sampling moments within each sampling time period obtained by sampling in the memory of the current control host, including but not limited to memory, memory eMMC, UFS, and EEPROM, etc., for subsequent data processing and operations.

[0054] Since there are many brands of structured light cameras and different cameras have different analysis algorithms for depth images, but they will all filter out suspected invalid parts such as those exceeding the parallax and the resolution of the structured light. For the accuracy of analysis and the convenience of subsequent measurement, the effective pixel amount of the depth map obtained by the detected depth camera is acquired; through the calculation inside the structured light camera, all invalid depth pixel points are obtained and discarded. For example, pixel points that cannot estimate the structured light inspection features such as overexposure, occlusion, being too close, or infrared reflection interference are discarded, and all effective pixel points are retained to reflect the operation features and oral environment features during measurement. The effective pixel amount is the number of pixel points retained; similarly, each depth map of 0.5 corresponds to an effective pixel amount, and each 10 seconds is a sampling time period, and the dataset of the effective pixel amount within the 10 seconds collected is still stored in the memory of the control host for subsequent joint analysis and processing.

[0055] Furthermore, the overexposure state of the structured light is monitored. In the embodiment of the present invention, an overexposure sensor is used to obtain overexposure data. The sampling frequency of the overexposure sensor is set to 10 Hz, that is, 10 data are collected per second. The 10 data obtained per second are processed. In this embodiment, variance is used to measure the change in the overexposure degree corresponding to each second. Then, the overexposure degree corresponding to this second can be obtained according to the variance of the 10 data obtained by the overexposure sensor per second. The sampling time period is also 10 seconds. Then, according to the overexposure degrees corresponding to each second within 10 seconds, a dataset corresponding to all overexposure degrees within the sampling time period can be obtained.

[0056] Based on the overexposure degree, effective pixel amount, and saliva interference rate corresponding to each sampling time period obtained above, the stability of the current structured light measurement and modeling process is analyzed to obtain the modeling stability corresponding to this sampling time period. Each sampling time period includes at least two sampling moments. The average value and the maximum value of the saliva interference rate at all sampling moments within the sampling time period are obtained; the ratio of the average value to the maximum value of the saliva interference rate within the sampling time period is the first ratio; the variance of the effective pixel amount at all sampling moments within the sampling time period is obtained, and the maximum value of the overexposure degree at all sampling moments within the sampling time period is obtained. An exponential function is constructed with the negative of the maximum value of the overexposure degree as the power exponent; the sum of the preset value and the variance of the effective pixel amount is calculated, and the ratio of the exponential function to the sum result is the second ratio; the product of the first ratio and the second ratio is the modeling stability. The specific calculation of the modeling stability is as follows:

[0057]

[0058] Among them, represents the modeling stability; represents the dataset corresponding to all saliva interference rates within the sampling time period; Represents the average value of all saliva interference rates within the sampling time period; Represents the maximum value of all saliva interference rates within the sampling time period; Represents the dataset corresponding to the effective pixel amounts of all depth maps within the sampling time period; Represents the variance calculation function; Is the preset value set in the embodiments of the present invention, used to avoid the situation where the denominator is 0; Represents the variance of all effective pixel amounts within the sampling time period. The smaller the value of the variance, the smaller the fluctuation of the effective pixel amounts of all depth maps within the sampling time period, the smoother the change of the effective pixel amounts, and the higher the modeling smoothness corresponding to the sampling time period; Represents the dataset corresponding to all overexposure degrees within the sampling time period; Represents the maximum value of the overexposure degree in the dataset corresponding to the overexposure degree. The larger the maximum value of the overexposure degree, the more serious the overexposure situation of the current structured light, the less conducive to the analysis of modeling, and the smaller the modeling smoothness; Represents the exponential function with the natural constant e as the base.

[0059] Is the first ratio, representing the difference between the average value and the maximum value of the saliva interference rate within the sampling time period. The larger this ratio, the closer the average value of the saliva interference rate is to the maximum value of the saliva interference rate within the sampling time period, the smaller the overall change of the saliva interference rate, and the larger the corresponding modeling smoothness; The larger the value of, the smaller the corresponding modeling smoothness, and the corresponding The smaller the value of, conversely, when The smaller the value of, then The larger the value of, the larger the corresponding modeling smoothness; Therefore, for the second ratio In, the numerator The larger the value of and the denominator The smaller the value of, the larger the corresponding modeling smoothness.

[0060] Thus, according to the calculation formula of the modeling smoothness, the corresponding modeling smoothness for each sampling time period is obtained. The larger the modeling smoothness, the better the smoothness of the modeling process under the sampling time period.

[0061] Step S200, perform clustering based on the modeling smoothness corresponding to different sampling time periods to obtain at least two clusters; obtain the infrared illuminance mean sequence of all sampling time periods within the cluster, and obtain the correlation index corresponding to the cluster based on the infrared illuminance mean sequence and the modeling smoothness of each sampling time period within the cluster.

[0062] To increase the reliability of data analysis, the analysis in the embodiments of the present invention is performed on multiple different patients, that is, corresponding to multiple modeling processes. Each modeling process includes multiple sampling time periods, and each sampling time period corresponds to a modeling smoothness. Analyzing the modeling smoothness within all sampling time periods can ensure that the smoothness characteristics under various different types are analyzed and processed. However, a large amount of data analysis will increase the computational amount. Therefore, to reduce the calculation, the sampling time periods with relatively close modeling smoothness characteristics are divided into one block for analysis.

[0063] Specifically, taking each sampling time period as a sample, each sample corresponds to a modeling smoothness. Cluster analysis is performed on all samples based on the modeling smoothness corresponding to each sample. In the embodiments of the present invention, the classic density clustering DBSCAN clustering algorithm is used. The clustering distance refers to the absolute value of the difference between the modeling smoothness corresponding to two samples. The search radius eps is defaulted to 0.1, and the minimum value within the cluster minpts is set to 5. Thus, all samples are divided into multiple clusters. For the rare case of isolated samples that appear during the clustering process, for the comprehensiveness of the analysis, the isolated samples form a separate cluster and serve as a new category for smoothness analysis. The DBSCAN clustering algorithm is a known technology in the art and will not be elaborated here.

[0064] Furthermore, obtain the infrared illuminance sequence of each sample within each cluster. If the infrared illuminance in the modeling environment increases, on the one hand, it may lead to a decrease in the measurement signal-to-noise ratio, and on the other hand, it will lead to an increase in the non-Lambert saliva interference rate in the oral cavity, which has a certain impact on the accuracy of the entire oral cavity modeling analysis. Obtain the infrared illuminance at each sampling moment in each sampling time period, and calculate the average value of the infrared illuminance at all sampling moments in each sampling time period. The average values of the infrared illuminance of all sampling time periods within the cluster form an infrared illuminance average value sequence. Based on the modeling smoothness of each sampling time period within the cluster, perform a descending order arrangement to obtain the corresponding modeling smoothness sequence of the cluster. The elements at the corresponding positions in the modeling smoothness sequence and the infrared illuminance average value sequence are the modeling smoothness and the infrared illuminance average value of the same sampling time period. Obtain the absolute value of the difference between the elements at the corresponding positions in the modeling smoothness sequence and the infrared illuminance average value sequence corresponding to each cluster, and sum up the absolute values of all corresponding position elements to obtain an accumulation result. Obtain the difference distance between the modeling smoothness sequence and the infrared illuminance average value sequence, and obtain a correlation index based on the ratio of the accumulation result to the difference distance.

[0065] For the collection of the environmental infrared illuminance during the modeling process, an infrared light sensor element can be directly provided externally, so that the infrared illuminance of the current environment can be directly detected. For example, by installing an infrared low-pass filter, the infrared radiation amount, that is, the infrared illuminance, can be collected. The specific cut-off wavelength depends on the infrared wavelength of the camera, generally 650 nm.

[0066] It should be noted that the infrared illuminance sequence of each sample is actually the infrared illuminance sequence within each sampling time period, and there is a corresponding infrared illuminance value at each sampling moment within each sampling time period. The sampling time period is also 10 seconds, and the sampling is performed every 0.5 seconds. Then, each sampling time period corresponds to an infrared illuminance sequence; and the obtained infrared illuminance sequences within each sampling time period are stored in the memory of the control host for subsequent analysis and processing.

[0067] Based on the analysis of all the infrared illuminance data corresponding to each sample within a cluster, first, all the samples within each cluster are sorted. The sorting rule is to sort them in descending order according to the modeling stability corresponding to each sample to obtain the modeling stability sequence corresponding to each cluster; then, the mean value of the infrared illuminance sequence corresponding to each sample is obtained. Since each sample corresponds to an infrared illuminance sequence, the mean value of all the infrared illuminance data in the infrared illuminance sequence is calculated to obtain the infrared illuminance mean value corresponding to each sample. The infrared illuminance mean values of all the samples within the cluster are arranged in sequence. The arrangement rule is based on the order of each sample in the modeling stability sequence corresponding to the cluster, and the infrared illuminance mean value sequence corresponding to all the samples within the cluster is obtained. The elements at the corresponding positions in the modeling stability sequence and the infrared illuminance mean value sequence are the modeling stability and the infrared illuminance mean value corresponding to the same sample.

[0068] Since the modeling stability sequence is sorted according to the data size, and the infrared illuminance mean value sequence is arranged based on the modeling stability sequence, there is no temporal correlation between the elements in the two sequences, only the correlation between the numerical distributions; the correlation index of the infrared light source interference is obtained according to the modeling stability sequence and the infrared illuminance mean value sequence corresponding to each cluster. The calculation method of the correlation index is as follows:

[0069]

[0070] Among them, represents the correlation index; represents the infrared illuminance mean value sequence; represents the modeling stability sequence; represents the dynamic time warping algorithm, which is used to obtain the difference distance between the two sequences; represents the difference distance between the infrared illuminance mean value sequence and the modeling stability sequence; is used to calculate the absolute value of the difference between the data at the corresponding positions in the infrared illuminance mean value sequence and the modeling stability sequence; represents the cumulative result of the absolute values of the differences between the data at the corresponding positions in the infrared illuminance mean value sequence and the modeling stability sequence; The closer the ratio is to 1, the more the absolute value of the difference between the infrared illumination mean sequence and the modeling stationarity sequence is. , and the difference distance between the infrared illumination mean series and the modeled stationary series The closer the value of is, the more synchronized the situation of each sample being interfered by infrared. Since the denominator is obtained by DTW dynamic time warping algorithm, the value is smaller than the numerator. When the ratio deviates from 1, the larger the value of this item, the more The correlation index obtained by calculation The smaller, that is The closer the ratio is to 1, the corresponding correlation index The bigger.

[0071] Based on the same method of calculating the correlation index of a cluster as mentioned above, the correlation index corresponding to the cluster can be obtained according to the modeling stationarity sequence and infrared illumination mean sequence corresponding to all samples in each cluster.

[0072] Step S300, obtain the membership of each sampling time period in the cluster, and obtain the modeling reliability based on the modeling stability and membership of each sampling time period and the correlation index of the cluster to which the sampling time period belongs; obtain the reconstruction validity of each sampling time period, and form a binary with the modeling reliability of the corresponding sampling time period.

[0073] Considering that each cluster includes multiple samples, but each sample in the cluster has a certain difference in the degree of membership to the cluster to which it belongs. The larger the degree of membership, the more the corresponding sample can represent the situation within the cluster to which it belongs, and the smaller the degree of membership, the weaker the representativeness of the sample in the cluster to which it belongs, and the less sociable it is compared with other samples. Therefore, when analyzing the modeling process of each sample corresponding to the sampling time period, the degree of membership of each sample in the cluster to which it belongs is used as an auxiliary judgment. The degree of membership of each sample in the cluster is obtained by the clustering distance between the sample and other samples in the cluster. The average value of the clustering distance between each sampling time period in the cluster and all sampling time periods in the cluster is obtained, and the average value is normalized to obtain the membership of the sampling time period. Taking sample A in any cluster as an example, the method for obtaining the degree of membership of sample A is:

[0074]

[0075] in, represents the membership degree of sample A; Represents sample A and its cluster Samples The cluster distance between them is the absolute value of the difference in modeling stability between samples; Indicates the number of all samples in the cluster to which sample A belongs; It represents the average value of the cluster distance between sample A and all samples in its cluster. The larger the average value, the greater the difference in modeling stability between sample A and other samples in its cluster. Sample A is less sociable in its cluster and has a smaller corresponding membership degree. represents a constant; Represents the cosine function, which is used to make the difference distance negatively correlated with the membership degree. Since the modeling stability corresponding to each sample is between 0 and 1, the average clustering distance between the modeling stability of sample A and other samples in its cluster must be between 0 and 1. The value of is 0- , ensuring that the value range of membership is 0-1, and The larger the value of The closer the value is to , the corresponding membership is closer to 0.

[0076] Based on the same method of obtaining the membership of sample A, the membership of each sample in each cluster is obtained. Then, based on the membership of each sample in the cluster and the correlation index of each cluster, the difference index of each sample in the cluster is obtained, which is used to analyze the modeling process corresponding to the current sample. The difference index is the product of the membership of each sample and the correlation index of its cluster, that is, the difference index is:

[0077]

[0078] in, represents the difference index; Indicates the membership of the sample; Indicates the correlation index of the cluster to which the sample belongs.

[0079] Membership The larger the value, the more representative the sample is within the cluster to which it belongs, and the correlation index of the cluster to which it belongs is The larger the value is, the more synchronously each sample in the cluster receives infrared interference. Therefore, the smaller the difference index obtained by multiplying the membership degree and the correlation index is, indicating that the sample is interfered by other factors other than external infrared interference during the modeling process, and the modeling process corresponding to the sample is not similar to that of other samples. This may be an abnormal measurement caused by operational reasons, such as dirty lenses.

[0080] Further, the modeling reliability of each sample is measured. During the modeling process, when the modeling stability corresponding to each sampling time period is stable and remains at a relatively high level, the corresponding modeling reliability is higher. At the same time, the modeling reliability is also relatively high under regular infrared illuminance changes. Therefore, the modeling reliability is obtained based on the modeling stability, membership degree, difference index, and current correlation index of each sample. Specifically, the calculation method for the modeling reliability of each sample is as follows:

[0081]

[0082] Among them, represents the modeling reliability; represents the modeling stability of the sample; represents the difference index of the sample; represents the correlation index of the cluster to which the sample belongs; represents the average value of the infrared illuminance sequence of the sample within the sampling time period; represents the hyperbolic tangent function, which is used for normalization processing; represents the function adjustment parameter, which is related to the magnitude of the reference infrared illuminance set by the implementer. The calculation method is , in the embodiment of the present invention, the reference infrared illuminance is set to 50 lux, and generally there will be no situation lower than 50 lux. Therefore, the corresponding .

[0083] It should be noted that during the calculation process of the modeling reliability, the modeling stability of the sample and the difference index of the sample are larger, indicating that the sample has greater representativeness in the cluster to which it belongs and the corresponding modeling process is stable. The corresponding modeling reliability is larger; the purpose of introducing the correlation index is that when the value of the correlation index is small, less consideration is given to the influencing factors of the infrared illuminance, that is, the value of is also small. On the contrary, when the value of the correlation index is large, it indicates that the corresponding modeling process of the sample is more affected by the infrared illuminance and the external interference is greater; The result of is used to correct the comprehensive result of the modeling reliability. Different oral cavities and different modeling processes have different sensitivities to infrared illuminance. The modeling reliability corrected by the infrared illuminance data is more accurate.

[0084] Furthermore, for the entire modeling process, from the perspective of playback, while obtaining the modeling reliability of each sample corresponding to the sampling time period, the point cloud modification area between the modeling reliability corresponding to the previous sampling time period of the current sampling time period and the modeling reliability of the current sampling time period can be obtained, that is, the area of the point cloud change. The point cloud modification area can be obtained by finding the changed points caused by the adjustment or optimization algorithm through the points between two moments, and the area formed by these changed points is the point cloud modification area. When the modeling reliability corresponding to the current sampling time period is high and the point cloud modification area is large, it means that the error in the early modeling process may be large. On the contrary, when the modeling reliability corresponding to the current sampling time period is high but the point cloud modification area is small, the error in the early modeling process is small. When the modeling reliability corresponding to the current sampling time period is low and the point cloud modification area is large, it means that the modeling process of this sampling time period is the initial stage of modeling, and when the modeling reliability corresponding to the current sampling time period is low and the point cloud modification area is also small, the modeling process of this sampling time period is the later stage of modeling.

[0085] Obtain the reconstruction effectiveness of the current sampling time period according to the point cloud modification area between the current sampling time period and its previous sampling time period. The reconstruction effectiveness is obtained by normalizing the point cloud modification area of the current sampling time period, and denote the reconstruction effectiveness as ; For the convenience of subsequent analysis and processing, the modeling reliability corresponding to each sampling time period and the reconstruction effectiveness are constructed into a binary tuple, and subsequent analysis is performed with the binary tuple corresponding to each sampling time period.

[0086] Step S400, input the binary tuple of the current sampling time period into the trained neural network to obtain the predicted binary tuple of each subsequent sampling time period, and regulate the three-dimensional model based on the predicted binary tuple.

[0087] For the convenience of subsequent obtaining and analyzing the modeling reliability and reconstruction effectiveness within different sampling time periods, in the embodiments of the present invention, the neural network is trained, and the state changes in the subsequent modeling process are determined through the trained neural network, that is, the predicted binary tuple of each sampling time period in the subsequent modeling process. The binary tuple includes the predicted modeling reliability and the predicted reconstruction effectiveness within the sampling time period; the training set of the neural network is composed of the binary tuples of different sampling time periods corresponding to different patients obtained. All the obtained samples are arranged according to the marked content as the change sequence of the modeling process. Each sample corresponds to a sampling time period, and the marked content refers to the modeling process and the modeling time corresponding to the sample. Therefore, the samples in the change sequence are arranged in sequence according to the modeling time under the same modeling process.

[0088] It should be noted that each sample in the change sequence corresponds to a binary tuple, that is, each sample corresponds to a reconstruction validity and a modeling reliability.

[0089] Obtain the weights of each sample. For any sample, accumulate the membership degrees corresponding to all samples in the change sequence where it is located to obtain an accumulation result. Use the accumulation result as the denominator for proportional normalization to normalize the membership degrees of each sample in the change sequence. The normalized membership degree of each sample is used as the weight corresponding to that sample. Further, accumulate the between-group weight differences corresponding to all samples in the change sequence corresponding to each modeling process and denote it as the accumulated difference. The between-group weight difference refers to the difference between the weights corresponding to the samples in the change sequence corresponding to each modeling process; and obtain the sum of the accumulated differences corresponding to all modeling processes, and normalize the accumulated difference of each modeling process according to the sum result of the accumulated differences to finally obtain the difference weight corresponding to each modeling process, denoted as .

[0090] When training the neural network using the training set, according to the overexposure degree, the effective pixel amount of the depth map in the 3D camera, the saliva interference rate, the infrared illuminance, and the correlation index corresponding to the cluster where the sample is located obtained in the above steps S100 - S300, and then obtain the modeling reliability and reconstruction validity of each sample. Input the binary tuples corresponding to all samples into the neural network for training. The neural network adopted in the embodiment of the present invention is an LSTM network, and the loss function adopts the mean square error loss function; during training, each sample in the change sequence corresponding to each modeling sample is sequentially used as the input of the network, and the predicted binary tuple of each sample is output; each sample obtains the corresponding weight based on membership degree normalization, and at the same time, the change sequence of each modeling process is normalized according to the between-group weight difference to obtain the difference weight. Therefore, the loss function is:

[0091]

[0092] Among them, represents the normalized membership degree corresponding to the th sample, that is, the weight corresponding to each sample. The greater the membership degree, the greater the corresponding weight; represents the difference weight of the change sequence of the modeling process where the sample is located, and the subscript n represents the change sequences corresponding to different modeling processes; represents the th sample's loss.

[0093] The training process of the neural network based on the loss function is a well-known existing method and will not be elaborated here. Training the LSTM network using the change sequences corresponding to different modeling processes ensures the diversity of the neural network learning and enables it to handle more modeling processes. Moreover, during subsequent analysis, the prediction binary tuple in the subsequent modeling process can be obtained according to the binary tuple composed of the existing modeling reliability and reconstruction effectiveness in the current modeling process, that is, predicting the prediction modeling reliability and prediction reconstruction effectiveness, which guarantees the accuracy of the prediction and enables advance prediction of subsequent situations, thus allowing for more timely adjustment and change of the modeling process.

[0094] When analyzing and modeling the oral cavity of a current patient, real-time information is input into the trained LSTM network model. The input is the binary tuple constructed from the modeling reliability and reconstruction effectiveness corresponding to the sample. The LSTM network model outputs the prediction binary tuple for a subsequent period of time, that is, the prediction modeling reliability and prediction reconstruction effectiveness. The specific prediction duration is determined according to the size of different convolutional kernels. In the embodiment of the present invention, it is in units of hours, and the LSTM network model outputs the prediction modeling reliability and prediction reconstruction effectiveness for the next half hour.

[0095] Analyze the prediction modeling reliability in the prediction binary tuple. Assume that the prediction modeling reliability is lower than the modeling reliability corresponding to the previous sampling time period, then intervene in the modeling process: First, set an intervention threshold for the reconstruction effectiveness. In this embodiment, the empirical value is set to 0.8. When the prediction reconstruction effectiveness is less than the intervention threshold of 0.8, divide the cost value of RANSAC or ICP for a future period of time by the prediction reconstruction effectiveness A, thereby intervening in the modeling process, that is, making the cost value larger, to avoid incorrect or unreliable point clouds from being incorporated into the model, thus improving stability. For example, when using the SALM system of RTABMAP, divide the cost function by the prediction modeling reliability H to improve stability and avoid overly abnormal point cloud merging results during this period, thereby improving the dental model accuracy. The specific processing process can be set by the implementer according to different situations and will not be elaborated here.

[0096] In summary, in the embodiment of the present invention, within a fixed sampling time period, the overexposure degree in the patient's oral cavity is collected and a depth map of the patient's oral cavity is obtained. Based on the depth map, the saliva interference rate and the effective pixel amount are obtained; the modeling smoothness is obtained according to the saliva interference rate, the overexposure degree, and the effective pixel amount; clustering is performed based on the modeling smoothness corresponding to different sampling time periods to obtain at least two clusters; the infrared illuminance mean sequence of all sampling time periods within the cluster is obtained, and the correlation index of the corresponding cluster is obtained based on the infrared illuminance mean sequence and the modeling smoothness of each sampling time period within the cluster; the membership degree of each sampling time period within the cluster is obtained, and the modeling reliability is obtained based on the modeling smoothness, the membership degree, and the correlation index of the cluster to which the sampling time period belongs for each sampling time period; the reconstruction effectiveness of each sampling time period is obtained, and a binary tuple is formed with the modeling reliability of the corresponding sampling time period; the binary tuple of the current sampling time period is input into the trained neural network to obtain the predicted binary tuple of each subsequent sampling time period, and the three-dimensional model is regulated based on the predicted binary tuple; the accuracy of analyzing the reconstruction situation of the three-dimensional model is improved, and the rationality of regulating the reconstruction of the three-dimensional model subsequently is ensured.

[0097] Based on the same inventive concept as the above method embodiment, the embodiment of the present invention further provides a three-dimensional model regulation system for orthodontics, and the system includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps in the above embodiment of a three-dimensional model regulation method for orthodontics are implemented, such as Figure 1 the steps shown. The above three-dimensional model regulation method for orthodontics has been described in detail in the above embodiments and will not be elaborated herein.

[0098] It should be noted that: the above sequence of the embodiments of the present invention is only for description and does not represent the superiority or inferiority of the embodiments. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0099] Each embodiment in this specification is described in a progressive manner, and the same or similar parts among the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

[0100] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A three-dimensional model control method for tooth correction, characterized in that: The method comprises the following steps: Within a fixed sampling time period, the overexposure degree in the patient's oral cavity is collected and a depth map of the patient's oral cavity is obtained, and a saliva interference rate and an effective pixel quantity are obtained based on the depth map; and a modeling stability is obtained according to the saliva interference rate, the overexposure degree and the effective pixel quantity; Clustering is performed based on the modeling stability corresponding to different sampling time periods to obtain at least two clusters; obtaining the infrared illumination mean sequence of all sampling time periods in the cluster, and obtaining the correlation index of the corresponding cluster based on the infrared illumination mean sequence and the modeling stability of each sampling time period in the cluster; Obtain the membership of each sampling time period in the cluster, and obtain the modeling reliability based on the modeling stability and membership of each sampling time period and the correlation index of the cluster to which the sampling time period belongs; obtain the reconstruction validity of each sampling time period, and form a binary with the modeling reliability of the corresponding sampling time period; Inputting the binary group of the current sampling time period into the trained neural network to obtain the predicted binary group of each subsequent sampling time period, and regulating the three-dimensional model based on the predicted binary group; The step of obtaining the modeling smoothness according to the saliva interference rate, the overexposure degree and the effective pixel quantity comprises: Each sampling time period includes at least two sampling moments, obtaining an average value and a maximum value of the saliva interference rate at all sampling moments in the sampling time period; calculating a ratio of the average value to the maximum value of the saliva interference rate in the sampling time period as a first ratio; Obtaining the variance of the effective pixel quantity at all sampling moments within the sampling time period, and obtaining the maximum value of the overexposure degree at all sampling moments within the sampling time period, and constructing an exponential function with the negative number of the maximum value of the overexposure degree as a power exponent; calculating the sum of the preset value and the variance of the effective pixel quantity, and the ratio of the exponential function to the sum is a second ratio; The product of the first ratio and the second ratio is the modeling smoothness.

2. A three-dimensional model control method for orthodontics according to claim 1, characterized in that: The step of obtaining the saliva interference rate and the effective pixel quantity based on the depth map includes: Acquire an overexposed area in the depth map, and record pixel points in the depth map other than the overexposed area as a to-be-processed set; Obtaining the median of the brightness values ​​of all pixels in the set to be processed, and arranging all the pixels in the set to be processed in descending order based on the brightness value of each pixel, selecting a set of pixels within a preset ratio after the descending order, and calculating the average of the brightness values ​​corresponding to all pixels in the selected set of pixels; The absolute value of the difference between the mean value and the median value is the saliva interference rate; Invalid depth pixels are obtained by internal calculation using a structured light camera that collects a depth map. Pixels in the depth map other than the invalid depth pixels are valid pixels, and the number of valid pixels is the effective pixel quantity.

3. A three-dimensional model control method for orthodontics according to claim 1, characterized in that: The step of clustering to obtain at least two clusters based on the modeling stationarities corresponding to different sampling time periods comprises: The absolute values ​​of the differences between the modeling stationarities corresponding to different sampling time periods are obtained as clustering distances, and the different sampling time periods are divided into at least two clusters based on the clustering distances.

4. A three-dimensional model control method for orthodontics according to claim 1, characterized in that: The step of obtaining the infrared illumination mean sequence of all sampling time periods in the cluster, and obtaining the correlation index of the corresponding cluster based on the infrared illumination mean sequence and the modeling stability of each sampling time period in the cluster, includes: Obtain the infrared illumination at each sampling moment in each sampling time period, and calculate the infrared illumination mean of all sampling moments in each sampling time period; the infrared illumination mean of all sampling time periods in the cluster constitutes an infrared illumination mean sequence; Arrange the modeling stability of each sampling time period in the cluster in descending order to obtain a modeling stability sequence corresponding to the cluster; the modeling stability sequence and the infrared illuminance mean sequence have corresponding position elements that are the modeling stability and infrared illuminance mean of the same sampling time period; Obtaining the absolute value of the difference between the modeling stationarity sequence corresponding to each cluster and the element at the corresponding position in the infrared illumination mean sequence, and summing the absolute values ​​of the differences of all the elements at the corresponding positions to obtain an accumulated result; The difference distance between the modeling stationarity sequence and the infrared illumination mean value sequence is obtained, and a correlation index is obtained based on the ratio of the accumulation result to the difference distance.

5. A three-dimensional model control method for orthodontics according to claim 3, characterized in that: The step of obtaining the degree of membership of each sampling time period in the cluster includes: The average value of the clustering distance between each sampling time period in the cluster and all the sampling time periods in the cluster to which it belongs is obtained, and the average value of the clustering distance is normalized to obtain the membership degree of the sampling time period.

6. A three-dimensional model control method for orthodontics according to claim 4, characterized in that: The step of obtaining the modeling reliability based on the modeling stability and membership of each sampling time period and the correlation index of the cluster to which the sampling time period belongs includes: The calculation formula for the modeling reliability is: in, Indicates the reliability of modeling; Indicates the modeling stationarity of the sampling period; represents the difference index of the sampling time period; Indicates the correlation index of the cluster to which the sampling time period belongs; Indicates the average infrared illumination value during the sampling period; represents the hyperbolic tangent function; Indicates function adjustment parameters; The difference index is the product of the membership degree of the corresponding sampling time period and the correlation index of the cluster to which the corresponding sampling time period belongs.

7. A three-dimensional model control method for orthodontics according to claim 1, characterized in that: The step of obtaining the reconstruction validity of each sampling time period includes: The point cloud modification area of ​​the modeling process corresponding to the previous sampling time period and the current sampling time period is obtained, and the point cloud modification area is normalized as the reconstruction validity of the current sampling time period.

8. A three-dimensional model control method for orthodontics according to claim 1, characterized in that: The training set of the neural network is a binary group of different patients' oral cavity in different sampling time periods; the weight of the loss function of the neural network is obtained by the membership degree of the corresponding sampling time period.

9. A three-dimensional model control system for orthodontics, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When executing the computer program, the processor implements the steps of the method described in any one of claims 1 to 8.

Images