Three-dimensional model regulation and control method and system for orthodontics

Abstract

The invention relates to the technical field of neural network models, in particular to a three-dimensional model regulation and control method and system for orthodontics. The method comprises the following steps: in a fixed sampling time period, acquiring saliva interference rate, overexposure degree and effective pixel quantity in the oral cavity of a patient at each sampling moment so as to obtain modeling stability; performing clustering according to the modeling stability corresponding to the plurality of sampling time periods to obtain at least two clusters; obtaining the relevance index of each cluster and the membership degree of each sampling time period in the cluster to which each sampling time period belongs, and further obtaining the modeling reliability degree of the corresponding sampling time period; obtaining reconstruction validity, constructing a two-tuple in combination with modeling reliability, obtaining a prediction two-tuple by using the trained neural network, and regulating and controlling the construction of the three-dimensional model based on the prediction two-tuple; the analysis accuracy in the modeling process is improved, and the model regulation and control effect is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of neural network models, and in particular to a three-dimensional model control method and system for orthodontics. Background Art

[0002] At present, digital dental models are a common and practical technology. The dental clinic uses an oral scanner to scan the teeth and then upload the data to the system. The designer designs the dental model on the computer based on the scanned data, and then prints it through 3D technology. Current mobile phones and other devices are equipped with structured light cameras, which can be used to quickly create dental models. Therefore, the difficulty of measurement will become lower and lower in the future, but the difficulty of modeling will gradually increase.

[0003] During the modeling process, the accuracy of the construction using a 3D structured light camera will be higher and faster than that of photos and videos. However, since the camera is active infrared, it is necessary to adjust the imprecise samples according to the modeling environment of the dental mold. Otherwise, the final 3D model reconstruction will be less accurate and the modeling effect will be poor due to the influence of the environment. Summary of the invention

[0004] In order to solve the problem of low accuracy and poor effect of the above-mentioned three-dimensional model reconstruction, the purpose of the present invention is to provide a three-dimensional model control method and system for tooth correction, and the technical solution adopted is as follows: In a first aspect, an embodiment of the present invention provides a three-dimensional model control method for orthodontics, the method comprising the following steps: Within a fixed sampling time period, the overexposure degree in the patient's oral cavity is collected and a depth map of the patient's oral cavity is obtained, and a saliva interference rate and an effective pixel quantity are obtained based on the depth map; and a modeling stability is obtained according to the saliva interference rate, the overexposure degree and the effective pixel quantity; Clustering is performed based on the modeling stability corresponding to different sampling time periods to obtain at least two clusters; obtaining the infrared illumination mean sequence of all sampling time periods in the cluster, and obtaining the correlation index of the corresponding cluster based on the infrared illumination mean sequence and the modeling stability of each sampling time period in the cluster; Obtain the membership of each sampling time period in the cluster, and obtain the modeling reliability based on the modeling stability and membership of each sampling time period and the correlation index of the cluster to which the sampling time period belongs; obtain the reconstruction validity of each sampling time period, and form a binary with the modeling reliability of the corresponding sampling time period; The binary group of the current sampling time period is input into the trained neural network to obtain the predicted binary group of each subsequent sampling time period, and the three-dimensional model is regulated based on the predicted binary group.

[0005] Preferably, the step of obtaining the saliva interference rate and the effective pixel quantity based on the depth map includes: Acquire an overexposed area in the depth map, and record pixel points in the depth map other than the overexposed area as a to-be-processed set; Obtaining the median of the brightness values ​​of all pixels in the set to be processed, and arranging all the pixels in the set to be processed in descending order based on the brightness value of each pixel, selecting a set of pixels within a preset ratio after the descending order, and calculating the average of the brightness values ​​corresponding to all pixels in the selected set of pixels; The absolute value of the difference between the mean value and the median value is the saliva interference rate; Invalid depth pixels are obtained by internal calculation using a structured light camera that collects a depth map. Pixels in the depth map other than the invalid depth pixels are valid pixels, and the number of valid pixels is the effective pixel quantity.

[0006] Preferably, the step of obtaining the modeling smoothness according to the saliva interference rate, the overexposure degree and the effective pixel quantity comprises: Each sampling time period includes at least two sampling moments, obtaining an average value and a maximum value of the saliva interference rate at all sampling moments in the sampling time period; calculating a ratio of the average value to the maximum value of the saliva interference rate in the sampling time period as a first ratio; Obtaining the variance of the effective pixel quantity at all sampling moments within the sampling time period, and obtaining the maximum value of the overexposure degree at all sampling moments within the sampling time period, and constructing an exponential function with the negative number of the maximum value of the overexposure degree as a power exponent; calculating the sum of the preset value and the variance of the effective pixel quantity, and the ratio of the exponential function to the sum is a second ratio; The product of the first ratio and the second ratio is the modeling smoothness.

[0007] Preferably, the step of clustering to obtain at least two clusters based on the modeling stationarities corresponding to different sampling time periods comprises: The absolute values ​​of the differences between the modeling stationarities corresponding to different sampling time periods are obtained as clustering distances, and the different sampling time periods are divided into at least two clusters based on the clustering distances.

[0008] Preferably, the step of obtaining the infrared illumination mean sequence of all sampling time periods in the cluster, and obtaining the correlation index of the corresponding cluster based on the infrared illumination mean sequence and the modeling stationarity of each sampling time period in the cluster includes: Obtain the infrared illumination at each sampling moment in each sampling time period, and calculate the infrared illumination mean of all sampling moments in each sampling time period; the infrared illumination mean of all sampling time periods in the cluster constitutes an infrared illumination mean sequence; Arrange the modeling stability of each sampling time period in the cluster in descending order to obtain a modeling stability sequence corresponding to the cluster; the modeling stability sequence and the infrared illuminance mean sequence have corresponding position elements that are the modeling stability and infrared illuminance mean of the same sampling time period; Obtaining the absolute value of the difference between the modeling stationarity sequence corresponding to each cluster and the element at the corresponding position in the infrared illumination mean sequence, and summing the absolute values ​​of the differences of all the elements at the corresponding positions to obtain an accumulated result; The difference distance between the modeling stationarity sequence and the infrared illumination mean value sequence is obtained, and a correlation index is obtained based on the ratio of the accumulation result to the difference distance.

[0009] Preferably, the step of obtaining the degree of membership of each sampling time period in the cluster includes: The average value of the clustering distance between each sampling time period in the cluster and all the sampling time periods in the cluster to which it belongs is obtained, and the average value of the clustering distance is normalized to obtain the membership degree of the sampling time period.

[0010] Preferably, the step of obtaining the modeling reliability based on the modeling stability and membership of each sampling time period and the correlation index of the cluster to which the sampling time period belongs includes: The calculation formula for the modeling reliability is: in, Indicates the reliability of modeling; Indicates the modeling stationarity of the sampling period; represents the difference index of the sampling time period; Indicates the correlation index of the cluster to which the sampling time period belongs; Indicates the average infrared illumination value during the sampling period; represents the hyperbolic tangent function; Indicates function adjustment parameters; The difference index is the product of the membership degree of the corresponding sampling time period and the correlation index of the cluster to which the corresponding sampling time period belongs.

[0011] Preferably, the step of obtaining the reconstruction validity of each sampling time period includes: The point cloud modification area of ​​the modeling process corresponding to the previous sampling time period and the current sampling time period is obtained, and the point cloud modification area is normalized as the reconstruction validity of the current sampling time period.

[0012] Preferably, the training set of the neural network is a binary group of oral cavity of different patients in different sampling time periods; the weight of the loss function of the neural network is obtained by the membership degree of the corresponding sampling time period.

[0013] In a second aspect, an embodiment of the present invention provides a three-dimensional model control system for orthodontics, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the processor implements the steps of the above-mentioned three-dimensional model control method for orthodontics.

[0014] The present invention has the following beneficial effects: the embodiment of the present invention performs analysis based on each sampling time period, and obtains the modeling stability of the sampling time period by obtaining various indicators within the sampling time period, and the various indicators of the sampling time period are obtained by combining the saliva interference rate, the effective pixel quantity and the overexposure degree of the image, and the three indicators are used to calculate the modeling stability of the corresponding sampling time period, and the combination of various information results is more reliable; further clustering is performed according to the modeling stability corresponding to different sampling time periods to obtain at least two clusters, and the correlation index of the modeling stability of each sampling time period in the cluster and the infrared illumination mean sequence corresponding to the cluster is obtained, which reflects the infrared interference suffered by the current modeling process; then the cluster is obtained The membership of each sampling time period is combined with the modeling stability and the correlation index of the cluster to which it belongs to obtain the modeling reliability. The modeling reliability is obtained from infrared interference, image features and the membership of the sampling time period itself, ensuring the credibility of the modeling reliability indicator. A tuple is formed by combining the reconstruction effectiveness corresponding to each sampling time period. The tuple is used as the whole input neural network to obtain the predicted tuple, which can timely and accurately obtain the modeling status of subsequent sampling time periods, and timely regulate the three-dimensional model reconstruction process based on the predicted tuple; it not only improves the accuracy of the modeling reliability analysis in the three-dimensional modeling process, but also can timely and effectively regulate the three-dimensional model, and the model construction effect is better. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 A flow chart of a three-dimensional model control method for orthodontics provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the three-dimensional model control method and system for tooth correction proposed by the present invention, its specific implementation method, structure, characteristics and effects, in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures, or characteristics in one or more embodiments may be combined in any suitable form.

[0018] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0019] The following is a detailed description of a three-dimensional model control method and system for orthodontics provided by the present invention in conjunction with the accompanying drawings.

[0020] See also Figure 1 , which shows a flow chart of a three-dimensional model control method for tooth correction provided by an embodiment of the present invention, the method comprising the following steps: Step S100, within a fixed sampling time period, collect the overexposure degree in the patient's mouth and obtain a depth map inside the patient's mouth, obtain the saliva interference rate and the effective pixel quantity based on the depth map; obtain the modeling stability according to the saliva interference rate, the overexposure degree and the effective pixel quantity.

[0021] Specifically, when analyzing and modeling the patient's oral cavity, the presence of saliva in the patient's oral cavity will cause more reflective interference in the oral cavity, which will cause more erroneous information for structured light measurement. Therefore, a method for measuring the non-Lambert saliva interference rate of structured light in the oral cavity is proposed in an embodiment of the present invention. The overexposed area in the depth map is obtained, and the pixels in the depth map except the overexposed area are recorded as a set to be processed; the median of the brightness values ​​of all pixels in the set to be processed is obtained, and all pixels in the set to be processed are arranged in descending order based on the brightness value of each pixel, a set of pixels within a preset ratio after the descending order is selected, and the average value of the corresponding brightness values ​​of all pixels in the selected set of pixels is calculated; the absolute value of the difference between the average value and the median value is the saliva interference rate; the specific measurement and evaluation method of the saliva interference rate is: A depth map of the inside of the patient's mouth at each sampling moment is obtained. Excessive saliva in the mouth will cause reflection interference, so when displayed in the image, an overexposed area will appear in the depth map. The overexposed area is the area where the pixel brightness exceeds the range. Generally, the pixel value range of the pixel is 0-255. Taking a conventional mobile phone as an example, it is generally believed that when the value of a pixel in all channels is 255, it is the pixel in the overexposed area. For a single-channel infrared camera, the pixel in the overexposed area is the pixel with a pixel value of 255 in a single channel. The depth map of the inside of the patient's mouth at each sampling moment is analyzed to obtain the overexposed pixels therein, and the connected domain analysis of all the overexposed pixels can obtain the corresponding overexposed area. The method of connected domain analysis is an existing well-known technology and will not be repeated here.

[0022] Furthermore, the overexposed area is expanded at each sampling moment. The expansion method in the embodiment of the present invention adopts a dilation algorithm to obtain the final overexposed connected domain. The dilation algorithm is a commonly used technology in image processing and will not be described in detail. When the depth map is taken, the places with more saliva will be refracted or scattered due to the light spots emitted by the structured light, thereby producing overexposed areas. The overexposed areas will affect the position estimation of the light spots, thereby producing large errors, which will have a great impact on the analysis of the situation in the patient's mouth. Therefore, the overexposed pixels in the overexposed connected domain are discarded, and other normal pixels in the depth map are analyzed, and the remaining pixel set after discarding all the overexposed pixels in the overexposed connected domain is recorded as the set to be processed.

[0023] The preset ratio is set to 30%, and the median of the brightness values ​​of all pixels in the set to be processed and the top 30% of pixels with larger brightness values ​​in the set to be processed are selected. That is, all pixels in the set to be processed are arranged in descending order according to the brightness value, and the top 30% of pixels are selected as top30%. The average brightness value of the top30% of pixels in the set to be processed is calculated; the corresponding saliva interference rate is obtained according to the average brightness value of the top30% of pixels in the set to be processed and the median brightness value of all pixels in the set to be processed; the specific calculation of the saliva interference rate is: in, represents the saliva interference rate; Represents a collection to be processed; Represents the median value of all pixels in the set to be processed; Represents the average brightness value of the top 30% of pixels in the set to be processed; Indicates absolute value calculation.

[0024] The larger the value of is, the more uneven the brightness is in the combination to be processed, and there is a large difference between the mean of the larger part of the brightness and the median of the overall brightness. Therefore, the more serious the saliva interference is, the greater the corresponding saliva interference rate is.

[0025] Since a depth map of the patient's mouth can be obtained at each sampling moment, each depth map corresponds to a saliva interference rate. In order to establish a time series analysis, in the embodiment of the present invention, every 10 seconds is set as a fixed sampling time period. During the sampling time period, image acquisition is performed at a sampling frequency of 2 Hz, that is, image acquisition is performed every 0.5 second as a sampling moment. Then, within the sampling time period of 10 seconds, corresponding depth maps at 20 sampling moments can be obtained, and corresponding saliva interference rates can be obtained based on the 20 depth maps. The sampling time period and sampling frequency can be set by the implementer according to different requirements.

[0026] It should be noted that the sampling start time is the start time of the patient's oral modeling, and the saliva interference rate collected at different sampling times in each sampling time period is stored in the memory of the current control host, including but not limited to memory, eMMC, UFS and EEPROM, etc., to facilitate subsequent data processing and calculation.

[0027] Since there are many brands of structured light cameras, different cameras have different analysis algorithms for depth images, but they will filter out suspected invalid parts such as those exceeding the parallax and the structured light resolution. In order to ensure the accuracy of the analysis and the convenience of subsequent measurement, the effective pixel quantity of the depth map obtained by the detected depth camera is obtained; through the calculation inside the structured light camera, all invalid depth pixels are obtained and discarded, such as pixels that cannot estimate the structured light inspection characteristics due to overexposure, occlusion, too close or infrared reflection interference, and all valid pixels are retained to reflect the operating characteristics and oral environment characteristics during measurement. The effective pixel quantity is the number of pixels retained; similarly, each 0.5 depth map corresponds to an effective pixel quantity, and each 10 seconds is a sampling period. The data set on the effective pixel quantity within 10 seconds collected is still stored in the memory of the control host for subsequent analysis and processing.

[0028] Furthermore, the overexposure state of the structured light is monitored. In the embodiment of the present invention, an overexposure sensor is used to obtain overexposure data. The sampling frequency of the overexposure sensor is set to 10 Hz, that is, 10 data are collected per second. The 10 data obtained per second are processed. In this embodiment, variance is used to measure the change in the overexposure degree corresponding to each second. The overexposure degree corresponding to this second can be obtained according to the variance of the 10 data obtained by the overexposure sensor per second. The sampling time period is also 10 seconds. According to the overexposure degree corresponding to each second in 10 seconds, a data set corresponding to all overexposure degrees in the sampling time period can be obtained.

[0029] According to the corresponding overexposure degree, effective pixel quantity and saliva interference rate in each sampling time period obtained above, the stability of the current structured light measurement and modeling process is analyzed to obtain the corresponding modeling stability in the sampling time period. Each sampling time period includes at least two sampling moments. The average value and maximum value of the saliva interference rate at all sampling moments in the sampling time period are obtained; the ratio of the average value to the maximum value of the saliva interference rate in the sampling time period is calculated as the first ratio; the variance of the effective pixel quantity at all sampling moments in the sampling time period is obtained, and the maximum value of the overexposure degree at all sampling moments in the sampling time period is obtained, and the negative number of the maximum value of the overexposure degree is used as the power exponent to construct an exponential function; the sum of the preset value and the variance of the effective pixel quantity is calculated, and the ratio of the exponential function to the sum result is the second ratio; the product of the first ratio and the second ratio is the modeling stability. The specific calculation of modeling stability is: in, Indicates the modeling stability; Represents the data set corresponding to all saliva interference rates during the sampling period; It represents the average value of all saliva interference rates during the sampling period; It represents the maximum value of all saliva interference rates during the sampling period; Represents the data set corresponding to the effective pixel quantity of all depth maps within the sampling time period; represents the variance calculation function; is a preset value set in the embodiment of the present invention, used to avoid the situation where the denominator is 0; It represents the variance of all effective pixels in the sampling period. The smaller the variance, the smaller the fluctuation of effective pixels in all depth maps in the sampling period. The more stable the change of effective pixels, the higher the modeling stability of the corresponding sampling period. Represents the data set corresponding to all overexposure levels within the sampling period; Indicates the maximum value of the overexposure degree in the data set corresponding to the overexposure degree. The larger the maximum value of the overexposure degree, the more serious the overexposure of the current structured light is, which is more unfavorable for modeling analysis and the lower the modeling stability is; Represents an exponential function with the natural constant e as the base.

[0030] is the first ratio, which indicates the difference between the average and maximum values ​​of the saliva interference rate during the sampling period. The larger the ratio is, the closer the average value of the saliva interference rate during the sampling period is to the maximum value of the saliva interference rate, and the smaller the change of the overall saliva interference rate is, and the greater the corresponding modeling stability is. The larger the value of is, the smaller the corresponding modeling stability is. The smaller the value of The smaller the value of The larger the value of is, the greater the corresponding modeling stability is; therefore, the second ratio In, molecules The larger the value of The smaller the value of is, the greater the corresponding modeling stability is.

[0031] Therefore, the corresponding modeling stability in each sampling time period is obtained according to the calculation formula of the modeling stability. The larger the modeling stability, the better the stability of the modeling process in the sampling time period.

[0032] Step S200, clustering is performed based on the modeling stability corresponding to different sampling time periods to obtain at least two clusters; the infrared illumination mean sequence of all sampling time periods in the cluster is obtained, and the correlation index of the corresponding cluster is obtained based on the infrared illumination mean sequence and the modeling stability of each sampling time period in the cluster.

[0033] In order to increase the reliability of data analysis, the analysis in the embodiment of the present invention is performed on multiple different patients, that is, corresponding to multiple modeling processes, each modeling process includes multiple sampling time periods, and each sampling time period corresponds to a modeling stability. The modeling stability in all sampling time periods is analyzed to ensure that the stability characteristics of each different type are analyzed and processed. However, a large amount of data analysis will increase the amount of calculation, so in order to reduce the calculation, the sampling time periods with a relatively close modeling stability feature are divided into one for analysis.

[0034] Specifically, each sampling time period is taken as a sample, and each sample corresponds to a modeling stationarity. All samples are clustered based on the modeling stationarity corresponding to each sample. The classic density clustering DBSCAN clustering algorithm is adopted in the embodiment of the present invention. The clustering distance refers to the absolute value of the difference between the modeling stationarities corresponding to two samples. The search radius eps defaults to 0.1, and the minimum value minpts within the cluster is set to 5. In this way, all samples are divided into multiple clusters. Isolated samples that appear in the clustering process are rare samples. For the comprehensiveness of the analysis, isolated samples are clustered separately as a new category of stationarity analysis. The DBSCAN clustering algorithm is an existing well-known technology and will not be described in detail.

[0035] Furthermore, the infrared illumination sequence of each sample in each cluster is obtained. If the infrared illumination of the modeling environment increases, on the one hand, the measurement signal-to-noise ratio may be reduced, and on the other hand, the non-Lambert saliva interference rate in the oral cavity will increase, which has a certain impact on the accuracy of the entire oral modeling analysis; the infrared illumination at each sampling moment in each sampling time period is obtained, and the infrared illumination mean of all sampling moments in each sampling time period is calculated; the infrared illumination means of all sampling time periods in the cluster constitute the infrared illumination mean sequence; based on the modeling stationarity of each sampling time period in the cluster, the modeling stationarity sequence corresponding to the cluster is obtained by arranging them in descending order; the modeling stationarity sequence and the corresponding position elements in the infrared illumination mean sequence are the modeling stationarity and infrared illumination mean of the same sampling time period; the absolute value of the difference between the modeling stationarity sequence corresponding to each cluster and the corresponding position elements in the infrared illumination mean sequence is obtained, and the absolute value of the difference of all corresponding position elements is summed to obtain the cumulative result; the difference distance between the modeling stationarity sequence and the infrared illumination mean sequence is obtained, and the correlation index is obtained based on the ratio of the cumulative result to the difference distance.

[0036] For the collection of ambient infrared illuminance during the modeling process, an infrared light-sensing element can be directly provided externally, so that the infrared illuminance of the current environment can be directly detected; for example, by installing an infrared low-pass filter, the infrared radiation amount, that is, the infrared illuminance, can be collected. The specific cut-off wavelength depends on the infrared wavelength of the camera, which is generally 650nm.

[0037] It should be noted that the infrared illuminance sequence of each sample is actually the infrared illuminance sequence within each sampling time period, and each sampling moment within each sampling time period corresponds to an infrared illuminance value. The sampling time period is also 10 seconds, and the sampling moment is collected every 0.5 seconds. Then, each sampling time period corresponds to an infrared illuminance sequence; and the infrared illuminance sequence obtained in each sampling time period is stored in the memory of the control host for subsequent analysis and processing.

[0038] Based on the analysis of all infrared illumination data corresponding to each sample in a cluster, firstly, all samples in each cluster are sorted. The sorting rule is to sort in descending order according to the modeling stationarity corresponding to each sample to obtain the modeling stationarity sequence corresponding to each cluster; then, the mean of the infrared illumination sequence corresponding to each sample is obtained. Since each sample corresponds to an infrared illumination sequence, the average of all infrared illumination data in the infrared illumination sequence is obtained to obtain the infrared illumination mean corresponding to each sample, and the infrared illumination mean of all samples in the cluster is arranged in sequence. The arrangement rule is to perform the order of each sample in the modeling stationarity sequence corresponding to the cluster, and obtain the infrared illumination mean sequence corresponding to all samples in the cluster. The elements in the corresponding positions in the modeling stationarity sequence and the infrared illumination mean sequence are the modeling stationarity and infrared illumination mean corresponding to the same sample.

[0039] Since the modeling stationary sequence is sorted according to the data size, and the infrared illumination mean sequence is arranged based on the arrangement of the modeling stationary sequence, there is no temporal correlation between the elements in the two sequences, only the correlation between the numerical distributions; the correlation index of the infrared light source interference is obtained according to the modeling stationary sequence and the infrared illumination mean sequence corresponding to each cluster. The calculation method of the correlation index is: in, Represents the correlation index; represents the infrared illumination mean value sequence; Represents the modeling stationary series; represents the dynamic time warping algorithm, which is used to obtain the difference distance between two sequences; Represents the difference distance between the infrared illumination mean series and the modeled stationarity series; Used to calculate the absolute value of the difference between the infrared illumination mean sequence and the modeling stationarity sequence at the corresponding position; It represents the cumulative result of the absolute value of the difference between the corresponding position data of the infrared illumination mean sequence and the modeled stationarity sequence; The closer the ratio is to 1, the more the absolute value of the difference between the infrared illumination mean sequence and the modeled stationarity sequence is. , and the difference distance between the infrared illumination mean series and the modeled stationary series The closer the value of is, the more synchronized the situation of each sample being interfered by infrared. Since the denominator is obtained by DTW dynamic time warping algorithm, the value is smaller than the numerator. When the ratio deviates from 1, the larger the value of this item, the more The correlation index obtained by calculation The smaller, that is The closer the ratio is to 1, the corresponding correlation index The bigger.

[0040] Based on the same method of calculating the correlation index of a cluster as mentioned above, the correlation index corresponding to the cluster can be obtained according to the modeling stationarity sequence and infrared illumination mean sequence corresponding to all samples in each cluster.

[0041] Step S300, obtain the membership of each sampling time period in the cluster, and obtain the modeling reliability based on the modeling stability and membership of each sampling time period and the correlation index of the cluster to which the sampling time period belongs; obtain the reconstruction validity of each sampling time period, and form a binary with the modeling reliability of the corresponding sampling time period.

[0042] Considering that each cluster includes multiple samples, but each sample in the cluster has a certain difference in the degree of membership to the cluster to which it belongs. The larger the degree of membership, the more the corresponding sample can represent the situation within the cluster to which it belongs, and the smaller the degree of membership, the weaker the representativeness of the sample in the cluster to which it belongs, and the less sociable it is compared with other samples. Therefore, when analyzing the modeling process of each sample corresponding to the sampling time period, the degree of membership of each sample in the cluster to which it belongs is used as an auxiliary judgment. The degree of membership of each sample in the cluster is obtained by the clustering distance between the sample and other samples in the cluster. The average value of the clustering distance between each sampling time period in the cluster and all sampling time periods in the cluster is obtained, and the average value is normalized to obtain the membership of the sampling time period. Taking sample A in any cluster as an example, the method for obtaining the degree of membership of sample A is: in, represents the membership degree of sample A; Represents sample A and its cluster Samples The cluster distance between them is the absolute value of the difference in modeling stability between samples; Indicates the number of all samples in the cluster to which sample A belongs; It represents the average value of the cluster distance between sample A and all samples in its cluster. The larger the average value, the greater the difference in modeling stability between sample A and other samples in its cluster. Sample A is less sociable in its cluster and has a smaller corresponding membership degree. represents a constant; Represents the cosine function, which is used to make the difference distance negatively correlated with the membership degree. Since the modeling stability corresponding to each sample is between 0 and 1, the average clustering distance between the modeling stability of sample A and other samples in its cluster must be between 0 and 1. The value of is 0- , ensuring that the value range of membership is 0-1, and The larger the value of The closer the value is to , the corresponding membership is closer to 0.

[0043] Based on the same method of obtaining the membership of sample A, the membership of each sample in each cluster is obtained. Then, based on the membership of each sample in the cluster and the correlation index of each cluster, the difference index of each sample in the cluster is obtained, which is used to analyze the modeling process corresponding to the current sample. The difference index is the product of the membership of each sample and the correlation index of its cluster, that is, the difference index is: in, represents the difference index; Indicates the membership of the sample; Indicates the correlation index of the cluster to which the sample belongs.

[0044] Membership The larger the value, the more representative the sample is within the cluster to which it belongs, and the correlation index of the cluster to which it belongs is The larger the value is, the more synchronously each sample in the cluster receives infrared interference. Therefore, the smaller the difference index obtained by multiplying the membership degree and the correlation index is, indicating that the sample is interfered by other factors other than external infrared interference during the modeling process, and the modeling process corresponding to the sample is not similar to that of other samples. This may be an abnormal measurement caused by operational reasons, such as dirty lenses.

[0045] Furthermore, the modeling reliability of each sample is measured. During the modeling process, when the modeling stability corresponding to each sampling time period is stable and remains at a high level, the corresponding modeling reliability is higher; at the same time, the modeling reliability under regular changes in infrared illumination is also higher. Therefore, the modeling reliability is obtained based on the modeling stability, membership, difference index and current correlation index of each sample. Specifically, the calculation method of the modeling reliability of each sample is: in, Indicates the reliability of modeling; Indicates the modeling stability of the sample; Represents the difference index of samples; Indicates the correlation index of the cluster to which the sample belongs; It represents the average value of the infrared illumination sequence of the sample within the sampling period; represents the hyperbolic tangent function, which is used for normalization; It represents the function adjustment parameter, which is related to the benchmark infrared illumination set by the implementer. The calculation method is: In the embodiment of the present invention, the reference infrared illuminance is set The value of is 50 lux, and generally there will be no situation where it is lower than 50 lux, so the corresponding .

[0046] It should be noted that in the calculation process of modeling reliability, the modeling stability of the sample And the difference index of the samples The larger the value, the more representative the sample is in the cluster to which it belongs and the modeling process corresponding to the sample is stable, and the corresponding modeling reliability is The larger the reference correlation index The purpose is to When the value of is small, the factors affecting infrared illumination are less considered, that is, The smaller the value of When the value of is large, it means that the modeling process corresponding to the sample is more affected by the infrared illumination and the external interference is greater; The results are used to correct the comprehensive results of modeling reliability. Different oral cavity and different modeling processes have different sensitivities to infrared illumination. The modeling reliability corrected by infrared illumination data is more accurate.

[0047] Furthermore, for the entire modeling process, from the perspective of playback, while obtaining the modeling reliability of each sample corresponding to the sampling time period, the point cloud modification area between the modeling reliability corresponding to the previous sampling time period of the current sampling time period and the modeling reliability of the current sampling time period can be obtained, that is, the area of ​​point cloud change. The point cloud modification area can be found through the points between two moments. The change points caused by adjustment or optimization algorithm, the area formed by these change points is the point cloud modification area; when the modeling reliability corresponding to the current sampling time period is high and the point cloud modification area is large, it means that the error in the early modeling process may be large, conversely, when the modeling reliability corresponding to the current sampling time period is high, but the point cloud modification area is small, the error in the early modeling process is small; when the modeling reliability corresponding to the current sampling time period is low and the point cloud modification area is large, it means that the modeling process of the sampling time period at this time is in the early stage of modeling, and when the modeling reliability corresponding to the current sampling time period is low and the point cloud modification area is small, the modeling process of the sampling time period at this time is in the late stage of modeling.

[0048] The reconstruction validity of the current sampling time period is obtained according to the point cloud modification area of ​​the current sampling time period and the previous sampling time period. The reconstruction validity is obtained after normalizing the point cloud modification area of ​​the current sampling time period. The reconstruction validity is recorded as ; In order to facilitate subsequent analysis and processing, the modeling reliability corresponding to each sampling time period is Reconstruction effectiveness A bigram is constructed, and subsequent analysis is performed based on the bigram corresponding to each sampling time period.

[0049] Step S400: Input the binary group of the current sampling time period into the trained neural network to obtain the predicted binary group of each subsequent sampling time period, and adjust the three-dimensional model based on the predicted binary group.

[0050] In order to facilitate the subsequent acquisition and analysis of the modeling reliability and reconstruction effectiveness in different sampling time periods, the neural network is trained in the embodiment of the present invention, and the state changes in the subsequent modeling process are determined by the trained neural network, that is, the predicted tuples for each sampling time period in the subsequent modeling process, and the tuples include the predicted modeling reliability and predicted reconstruction effectiveness in the sampling time period; the training set of the neural network is composed of tuples of different sampling time periods corresponding to different patients. All the obtained samples are arranged according to the marked content as a change sequence of the modeling process. Each sample corresponds to a sampling time period. The marked content refers to the modeling process and modeling time corresponding to the sample. Therefore, the arrangement of samples in the change sequence is obtained according to the same modeling process and arranged in sequence according to the modeling time.

[0051] It should be noted that each sample in the change sequence corresponds to a binary group, that is, each sample corresponds to a reconstruction validity and modeling reliability.

[0052] The weight of each sample is obtained. For any sample, the corresponding membership of all samples in the change sequence in which it is located is accumulated to obtain the accumulated result. The accumulated result is used as the denominator for proportional normalization to normalize the membership of each sample in the change sequence. The normalized membership of each sample is used as the weight corresponding to the sample. Furthermore, the inter-group weight differences corresponding to all samples in the change sequence corresponding to each modeling process are accumulated and recorded as the accumulated difference. The inter-group weight difference refers to the difference between the corresponding weights of the samples in the change sequence corresponding to each modeling process; and the accumulated differences corresponding to all modeling processes are obtained and summed. According to the sum of the accumulated differences, the accumulated differences of each modeling process are normalized to finally obtain the difference weight corresponding to each modeling process and recorded as .

[0053] When the neural network is trained using the training set, the overexposure degree corresponding to each sample, the effective pixel quantity of the depth map in the 3D camera, the saliva interference rate, the infrared illumination, and the correlation index corresponding to the cluster where the sample is located are obtained according to the above steps S100-S300, and then the modeling reliability and reconstruction effectiveness of each sample are obtained, and the binary groups corresponding to all samples are input into the neural network for training. The neural network used in the embodiment of the present invention is an LSTM network, and the loss function adopts a mean square error loss function; during training, each sample in the change sequence corresponding to each modeling sample is sequentially used as the input of the network, and the predicted binary group of each sample is output; each sample is normalized based on the membership degree to obtain the corresponding weight, and at the same time, the change sequence of each modeling process is normalized according to the weight difference between groups to obtain the difference weight, so the loss function is: in, Indicates The normalized membership degree corresponding to each sample is the weight corresponding to each sample. The larger the membership degree, the larger the corresponding weight. It represents the difference weight of the change sequence of the modeling process in which the sample is located. The subscript n represents the change sequence corresponding to different modeling processes. Indicates The loss of samples.

[0054] The training process of the neural network based on the loss function is an existing well-known method and will not be elaborated in detail. The LSTM network is trained using the change sequences corresponding to different modeling processes to ensure the diversity of neural network learning and to be able to cope with modeling processes for more conditions. In subsequent analysis, the predicted tuples in the subsequent modeling process can be obtained based on the tuples composed of the existing modeling reliability and reconstruction effectiveness in the current modeling process, that is, the predicted modeling reliability and the predicted reconstruction effectiveness are predicted, which ensures the accuracy of the prediction, and predicts subsequent situations in advance, so as to adjust and change the modeling process more timely.

[0055] When analyzing and modeling the current patient's oral cavity, real-time information is input into the trained LSTM network model. The input is a tuple constructed by the modeling reliability and reconstruction effectiveness corresponding to the sample. The trained LSTM network model outputs a predicted tuple for a subsequent period of time, that is, the predicted modeling reliability and predicted reconstruction effectiveness. The specific prediction duration is determined according to different convolution kernel sizes. In the embodiment of the present invention, the unit is hourly, and the LSTM network model outputs the predicted modeling reliability and predicted reconstruction effectiveness for the next half hour.

[0056] The predictive modeling reliability in the predicted binary group is analyzed. Assuming that the predictive modeling reliability is lower than the modeling reliability corresponding to the previous sampling time period, the modeling process is intervened: first, an intervention threshold is set for the reconstruction effectiveness. In this embodiment, the empirical value is set to 0.8; when the predicted reconstruction effectiveness is less than the intervention threshold 0.8, the cost value of RANSAC or ICP for a period of time in the future is divided by the predicted reconstruction effectiveness A, thereby intervening in the modeling process, that is, making the cost value larger to avoid erroneous or unreliable point clouds from being combined with the model, thereby improving stability; for example, when using the SALM system of RTABMAP, the cost function is divided by the predictive modeling reliability H to improve stability and avoid excessively abnormal point cloud merging results during this period, thereby improving the accuracy of the dental model. The specific processing process can be set by the implementer according to different situations and will not be repeated here.

[0057] In summary, the embodiment of the present invention collects the overexposure degree in the patient's mouth and obtains the depth map of the inside of the patient's mouth within a fixed sampling time period, and obtains the saliva interference rate and the effective pixel amount based on the depth map; obtains the modeling stability according to the saliva interference rate, the overexposure degree and the effective pixel amount; clusters at least two clusters based on the modeling stability corresponding to different sampling time periods; obtains the infrared illuminance mean sequence of all sampling time periods in the cluster, and obtains the correlation index of the corresponding cluster based on the infrared illuminance mean sequence and the modeling stability of each sampling time period in the cluster; obtains the membership of each sampling time period in the cluster, and obtains the modeling reliability based on the modeling stability, membership of each sampling time period and the correlation index of the cluster to which the sampling time period belongs; obtains the reconstruction effectiveness of each sampling time period, and forms a binary with the modeling reliability of the corresponding sampling time period; inputs the binary of the current sampling time period into the trained neural network to obtain the predicted binary of each subsequent sampling time period, and regulates the three-dimensional model based on the predicted binary; improves the accuracy of the analysis of the reconstruction of the three-dimensional model, and ensures the rationality of the subsequent regulation of the reconstruction of the three-dimensional model.

[0058] Based on the same inventive concept as the above method embodiment, the present invention also provides a three-dimensional model control system for orthodontics, the system comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps in the above three-dimensional model control method embodiment for orthodontics are implemented, for example Figure 1 The three-dimensional model control method for tooth correction has been described in detail in the above embodiment and will not be repeated here.

[0059] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0060] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0061] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A three-dimensional model control method for tooth correction, characterized in that: The method comprises the following steps: Within a fixed sampling time period, the overexposure degree in the patient's oral cavity is collected and a depth map of the patient's oral cavity is obtained, and a saliva interference rate and an effective pixel quantity are obtained based on the depth map; and a modeling stability is obtained according to the saliva interference rate, the overexposure degree and the effective pixel quantity; Clustering is performed based on the modeling stability corresponding to different sampling time periods to obtain at least two clusters; obtaining the infrared illumination mean sequence of all sampling time periods in the cluster, and obtaining the correlation index of the corresponding cluster based on the infrared illumination mean sequence and the modeling stability of each sampling time period in the cluster; Obtain the membership of each sampling time period in the cluster, and obtain the modeling reliability based on the modeling stability and membership of each sampling time period and the correlation index of the cluster to which the sampling time period belongs; obtain the reconstruction validity of each sampling time period, and form a binary with the modeling reliability of the corresponding sampling time period; Inputting the binary group of the current sampling time period into the trained neural network to obtain the predicted binary group of each subsequent sampling time period, and regulating the three-dimensional model based on the predicted binary group; The step of obtaining the modeling smoothness according to the saliva interference rate, the overexposure degree and the effective pixel quantity comprises: Each sampling time period includes at least two sampling moments, obtaining an average value and a maximum value of the saliva interference rate at all sampling moments in the sampling time period; calculating a ratio of the average value to the maximum value of the saliva interference rate in the sampling time period as a first ratio; Obtaining the variance of the effective pixel quantity at all sampling moments within the sampling time period, and obtaining the maximum value of the overexposure degree at all sampling moments within the sampling time period, and constructing an exponential function with the negative number of the maximum value of the overexposure degree as a power exponent; calculating the sum of the preset value and the variance of the effective pixel quantity, and the ratio of the exponential function to the sum is a second ratio; The product of the first ratio and the second ratio is the modeling smoothness.

2. A three-dimensional model control method for orthodontics according to claim 1, characterized in that: The step of obtaining the saliva interference rate and the effective pixel quantity based on the depth map includes: Acquire an overexposed area in the depth map, and record pixel points in the depth map other than the overexposed area as a to-be-processed set; Obtaining the median of the brightness values ​​of all pixels in the set to be processed, and arranging all the pixels in the set to be processed in descending order based on the brightness value of each pixel, selecting a set of pixels within a preset ratio after the descending order, and calculating the average of the brightness values ​​corresponding to all pixels in the selected set of pixels; The absolute value of the difference between the mean value and the median value is the saliva interference rate; Invalid depth pixels are obtained by internal calculation using a structured light camera that collects a depth map. Pixels in the depth map other than the invalid depth pixels are valid pixels, and the number of valid pixels is the effective pixel quantity.

3. A three-dimensional model control method for orthodontics according to claim 1, characterized in that: The step of clustering to obtain at least two clusters based on the modeling stationarities corresponding to different sampling time periods comprises: The absolute values ​​of the differences between the modeling stationarities corresponding to different sampling time periods are obtained as clustering distances, and the different sampling time periods are divided into at least two clusters based on the clustering distances.

4. A three-dimensional model control method for orthodontics according to claim 1, characterized in that: The step of obtaining the infrared illumination mean sequence of all sampling time periods in the cluster, and obtaining the correlation index of the corresponding cluster based on the infrared illumination mean sequence and the modeling stability of each sampling time period in the cluster, includes: Obtain the infrared illumination at each sampling moment in each sampling time period, and calculate the infrared illumination mean of all sampling moments in each sampling time period; the infrared illumination mean of all sampling time periods in the cluster constitutes an infrared illumination mean sequence; Arrange the modeling stability of each sampling time period in the cluster in descending order to obtain a modeling stability sequence corresponding to the cluster; the modeling stability sequence and the infrared illuminance mean sequence have corresponding position elements that are the modeling stability and infrared illuminance mean of the same sampling time period; Obtaining the absolute value of the difference between the modeling stationarity sequence corresponding to each cluster and the element at the corresponding position in the infrared illumination mean sequence, and summing the absolute values ​​of the differences of all the elements at the corresponding positions to obtain an accumulated result; The difference distance between the modeling stationarity sequence and the infrared illumination mean value sequence is obtained, and a correlation index is obtained based on the ratio of the accumulation result to the difference distance.

5. A three-dimensional model control method for orthodontics according to claim 3, characterized in that: The step of obtaining the degree of membership of each sampling time period in the cluster includes: The average value of the clustering distance between each sampling time period in the cluster and all the sampling time periods in the cluster to which it belongs is obtained, and the average value of the clustering distance is normalized to obtain the membership degree of the sampling time period.

6. A three-dimensional model control method for orthodontics according to claim 4, characterized in that: The step of obtaining the modeling reliability based on the modeling stability and membership of each sampling time period and the correlation index of the cluster to which the sampling time period belongs includes: The calculation formula for the modeling reliability is: in, Indicates the reliability of modeling; Indicates the modeling stationarity of the sampling period; represents the difference index of the sampling time period; Indicates the correlation index of the cluster to which the sampling time period belongs; Indicates the average infrared illumination value during the sampling period; represents the hyperbolic tangent function; Indicates function adjustment parameters; The difference index is the product of the membership degree of the corresponding sampling time period and the correlation index of the cluster to which the corresponding sampling time period belongs.

7. A three-dimensional model control method for orthodontics according to claim 1, characterized in that: The step of obtaining the reconstruction validity of each sampling time period includes: The point cloud modification area of ​​the modeling process corresponding to the previous sampling time period and the current sampling time period is obtained, and the point cloud modification area is normalized as the reconstruction validity of the current sampling time period.

8. A three-dimensional model control method for orthodontics according to claim 1, characterized in that: The training set of the neural network is a binary group of different patients' oral cavity in different sampling time periods; the weight of the loss function of the neural network is obtained by the membership degree of the corresponding sampling time period.

9. A three-dimensional model control system for orthodontics, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When executing the computer program, the processor implements the steps of the method described in any one of claims 1 to 8.

Images