Method for accurately matching CBCT data and STL data

Abstract

The invention relates to the technical field of data matching, and discloses a CBCT data and STL data accurate matching method, and the method comprises the steps: selecting an X-ray dose, a scanning angle and a resolution ratio according to different application scenes, and carrying out the preprocessing of a collected CBCT image; an STL model of the patient is obtained from a 3D scanner, noise and unnecessary details in the model are removed, and key feature points of the model are recognized and marked; a large number of CBCT images and corresponding STL models are collected, a training data set is constructed, and feature extraction is performed on the CBCT images by using a convolutional neural network. According to the method for accurately matching the CBCT data and the STL data, the X-ray dose is reasonably selected, so that the image quality can be ensured, and the radiation exposure risk of a patient can be reduced; by selecting a proper scanning angle and resolution, a complete anatomical region can be covered, meanwhile, it is ensured that details of the image are clear, and the high-quality STL model provides accurate reference for subsequent matching.

Description

Technical Field

[0001] The present invention relates to the technical field of data matching, and in particular to a method for accurately matching CBCT data with STL data. Background Art

[0002] Data matching refers to the process of finding a correspondence between two or more data sets. This correspondence can be based on the consistency of certain features or attributes, or it can be based on the similarity calculated by an algorithm. Data matching has a wide range of applications in many fields, especially in medicine, computer vision, geographic information systems (GIS), finance, and marketing. However, in the existing CBCT image acquisition process, due to problems such as noise, insufficient contrast, and blurred edges, the image quality is poor, which affects the subsequent matching accuracy. In addition, in the acquisition process of the existing STL model, due to the large amount of noise and unnecessary details, the model is highly complex, which increases the difficulty and amount of calculation for matching. At the same time, in the feature extraction and matching process, the existing matching method has low matching accuracy due to insufficient feature extraction and the limitations of the matching algorithm, which cannot meet the high-precision requirements of clinical applications. Summary of the invention

[0003] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0004] To achieve the above object, the present invention provides the following technical solutions:

[0005] A method for accurately matching CBCT data with STL data, the method comprising: selecting X-ray dose, scanning angle and resolution according to different application scenarios, and preprocessing the collected CBCT images; obtaining the patient's STL model from a 3D scanner, removing noise and unnecessary details in the model, and identifying and marking key feature points of the model; collecting a large number of CBCT images and corresponding STL models, building a training data set, and using a convolutional neural network to extract features from the CBCT images; inputting the collected CBCT images into a trained deep learning model to obtain preliminary matching results, and dynamically adjusting the matching strategy according to the patient's condition; using a standard evaluation indicator, root mean square error, to quantitatively evaluate the matching results, and feeding the evaluation results back to the deep learning model to continuously optimize the performance of the model.

[0006] As a further solution of the present invention: the X-ray dose, scanning angle and resolution are selected according to different application scenarios, and the steps include: for oral and maxillofacial surgery, a lower dose is used, and for complex craniofacial structures, the dose can be appropriately increased; in order to cover the complete anatomical area, a scanning angle of 180° to 360° is selected, for smaller areas, 180° scanning is selected; for larger areas, 360° scanning is used; for fine structures, high resolution is used, and for a large range of structures, medium resolution can be used.

[0007] As a further solution of the present invention: the preprocessing of the collected CBCT image includes the steps of using an adaptive filter to perform denoising on the CBCT image, and the expression is:

[0008]

[0009] Among them, I(x,y) is the original image, I denoised (x, y) is the denoised image, σ is the standard deviation of the noise, k is the adjustment parameter, is the image gradient, is the Laplace operator;

[0010] The histogram equalization method is used to enhance the contrast of the image. The expression is:

[0011] I enhanced (x,y)=T(I(x,y));

[0012] Where T is the histogram equalization transformation function, I(x,y) is the denoised image, and I enhanced (x, y) is the image after contrast enhancement;

[0013] Use the Canny edge detection algorithm to identify the edges in the image. The expression is:

[0014]

[0015] Among them, E(x,y) is the edge detection result, G x (x,y) and G y (x,y) is the derivative of the Gaussian filter, and is the gradient of the image in the x and y directions.

[0016] As a further solution of the present invention: the steps of obtaining the STL model of the patient from the 3D scanner, removing noise and unnecessary details in the model and identifying and marking key feature points of the model include:

[0017] Use a high-precision 3D scanner to scan the relevant parts of the patient and obtain the patient's STL model;

[0018] The STL model is denoised using a Laplace smoothing filter to smooth the model surface and remove small-scale noise;

[0019] Use curvature-based simplification algorithms to remove unnecessary details from the model;

[0020] Harris corner detection algorithm is used to identify the key feature points of the model.

[0021] As a further solution of the present invention: the steps of collecting a large number of CBCT images and corresponding STL models to construct a training data set include:

[0022] Collect a large number of CBCT images from multiple medical institutions and obtain STL models corresponding to the CBCT images from a 3D scanner;

[0023] The collected CBCT images and STL models were preliminarily registered to ensure that they were spatially aligned;

[0024] The registered CBCT images and STL models were aligned to form a one-to-one paired dataset.

[0025] As a further solution of the present invention: the method of using a convolutional neural network to extract features from CBCT images includes the following steps: designing a deep convolutional neural network model to extract features of CBCT images; using feature fusion technology to fuse the features of the CBCT images with the features of the STL model to form a joint feature vector, which is expressed as:

[0026] F joint =α·F CBCT +β·F STL ;

[0027] Among them, F joint is the joint eigenvector, F CBCT is the feature vector of CBCT image, F STL is the feature vector of the STL model, α and β are the weight coefficients of feature fusion;

[0028] A composite loss function is designed to combine the mean square error and the structural similarity index to evaluate the matching accuracy of the model. The expression is:

[0029] L = λ1·MSE(F joint ,F target )+λ2·(1-SSIM(F joint ,F target ));

[0030] Among them, L is the composite loss function, MSE is the mean square error, SSIM is the structural similarity index, and F joint is the joint eigenvector, F target is the target feature vector, λ1 and λ2 are the weight coefficients of the loss function.

[0031] As a further solution of the present invention: the collected CBCT image is input into the trained deep learning model to obtain a preliminary matching result, and the matching strategy is dynamically adjusted according to the patient's condition. The steps include: inputting the collected CBCT image into the trained deep learning model; the model extracts the features of the CBCT image through a convolutional neural network, and matches it with the pre-extracted STL model features to generate a preliminary matching result; using a dynamic adjustment function to adjust the weight of the matching strategy according to the patient's characteristics, the expression is:

[0032] M final =M initial +γ·f(P,M initial );

[0033] Among them, M final is the final matching result, M initial is the preliminary matching result, P is the patient feature vector, γ is the adjustment coefficient, f(P,M initial ) is a dynamic adjustment function;

[0034] Dynamic adjustment function f(P,M initial ) Generate adjustment terms by linearly combining patient characteristics and preliminary matching results, expressed as:

[0035] f(P,M initial )=w1·P1+w2·P2+…+w n ·P n +b·M initial ;

[0036] Among them, P1, P2…P n are the components of the patient’s feature vector, w1, w2, ... n is the corresponding weight, and b is the weight of the preliminary matching result.

[0037] As a further solution of the present invention: the matching results are quantitatively evaluated using the standard evaluation indicator root mean square error, and the evaluation results are fed back to the deep learning model to continuously optimize the performance of the model. The steps include using the root mean square error as an evaluation indicator to quantitatively evaluate the preliminary matching results and the final matching results. The expression is:

[0038]

[0039] Among them, RMSE is the root mean square error, M pred (i) is the predicted matching result, M true (i) is the actual matching result, N is the number of matching points;

[0040] The evaluation results are fed back into the deep learning model, and the model parameters are updated through the back propagation algorithm. The gradient descent method is used to update the model parameters to reduce the root mean square error, which is expressed as:

[0041]

[0042] Among them, θ new is the updated model parameter, θ old is the current model parameter, η is the learning rate, is the gradient of the root mean square error with respect to the model parameters.

[0043] The present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the method for accurate matching of CBCT data and STL data as described in the first aspect of the present invention is implemented.

[0044] The present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, any step of the method for accurate matching of CBCT data and STL data as described in the first aspect of the present invention is implemented. Compared with the prior art, the present invention has the following beneficial effects:

[0045] By rationally selecting the X-ray dose, we can ensure image quality and reduce the patient's radiation exposure risk. By selecting the appropriate scanning angle and resolution, we can cover the complete anatomical area while ensuring that the details of the image are clear. The high-quality STL model provides an accurate reference for subsequent matching. De-noising and detail removal ensure the simplicity and accuracy of the model. The marking of key feature points provides important positioning information for the matching process. A large number of CBCT images and corresponding STL models are collected from multiple medical institutions for preliminary registration to form a one-to-one paired data set. The convolutional neural network is used to extract the features of the CBCT images and perform feature fusion. Its high-quality training data set provides rich samples for the training of deep learning models. The convolutional neural network can capture the complex features in the image. The feature fusion technology further enhances the representativeness of the features. The preliminary matching results provide a basis for subsequent adjustments. The dynamic adjustment strategy makes the matching results more in line with the patient's specific situation and improves the personalization and accuracy of the matching. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1Schematic diagram of the process of accurately matching CBCT data with STL data. DETAILED DESCRIPTION

[0047] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and understandable, the specific implementation modes of the present invention are described in detail below in conjunction with the accompanying drawings.

[0048] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0049] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0050] Example

[0051] See also Figure 1 , is an embodiment of the present invention, which provides a method for accurately matching CBCT data with STL data, including:

[0052] For patients undergoing oral and maxillofacial surgery, a lower X-ray dose (0.05 mGy) was used, a scanning angle of 180° was selected, and a high resolution (0.1 mm) was used for scanning. For patients with complex craniofacial structures, a higher X-ray dose (0.2 mGy) was used, a scanning angle of 360° was selected, and a medium resolution (0.3 mm) was used for scanning. A high-precision 3D scanner was used to scan the patient's oral cavity or craniofacial area to obtain an STL model.

[0053] Adaptive filters are used to denoise CBCT images, histogram equalization is used to enhance image contrast, and Canny edge detection algorithm is used to identify edges in images.

[0054] The STL model is denoised using a Laplace smoothing filter to smooth the model surface and remove small-scale noise. The curvature-based simplification algorithm is used to remove unnecessary details in the model. The key feature points of the model are identified using the Harris corner detection algorithm.

[0055] A large number of CBCT images and corresponding STL models were collected from multiple medical institutions to ensure the diversity and representativeness of the data. The collected CBCT images and STL models were preliminarily registered to ensure that the two were spatially aligned. The registered CBCT images and STL models were aligned to form a one-to-one paired data set.

[0056] A deep convolutional neural network model is designed to extract the features of CBCT images. The features of CBCT images are fused with the features of STL models using feature fusion technology to form a joint feature vector. A composite loss function is designed to combine the mean square error and the structural similarity index to evaluate the matching accuracy of the model.

[0057] The acquired CBCT images are input into the trained deep learning model to generate preliminary matching results. A dynamic adjustment function is used to adjust the weight of the matching strategy according to the patient characteristics. The dynamic adjustment function generates adjustment items by linearly combining the patient characteristics and the preliminary matching results.

[0058] The root mean square error (RMSE) is used as the evaluation indicator to quantitatively evaluate the preliminary matching results and the final matching results. The evaluation results are fed back to the deep learning model and the model parameters are updated through the back propagation algorithm.

[0059] This embodiment also provides a computer device, which is applicable to the method for accurately matching CBCT data with STL data, and includes: a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions to implement the method for accurately matching CBCT data with STL data proposed in the above embodiment.

[0060] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0061] The present embodiment further provides a storage medium on which a computer program is stored. When the program is executed by a processor, the method for realizing accurate matching of CBCT data and STL data proposed in the above embodiment is realized. The storage medium can be realized by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable red-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0062] Table: Comparison of experimental results of CBCT data and STL data precise matching method

[0063]

[0064]

[0065] Through the analysis of the above experimental results, it can be clearly seen that the invention has innovation and superiority in the precise matching of CBCT data and STL data. The specific analysis is as follows:

[0066] By using an adaptive filter to denoise the CBCT images, a very high denoising effect was achieved. For oral patients, the denoising effect reached 94% to 96%, and for craniofacial patients, the denoising effect reached 91% to 93%. The results are much higher than the denoising effect of the prior art (about 80%), indicating that the denoising method of the present invention can effectively reduce the noise in the image and improve the image quality.

[0067] The key feature points of the model were identified using the Harris corner detection algorithm, and an extremely high feature point recognition rate was achieved. For oral patients, the feature point recognition rate reached 97% to 99%, and for craniofacial patients, the feature point recognition rate reached 95% to 97%. The results were significantly better than the feature point recognition rate of the prior art (about 85%), indicating that the feature point recognition method of the present invention can more accurately identify the key feature points in the model, providing a reliable basis for subsequent matching.

[0068] By using a deep convolutional neural network and a dynamic adjustment function, high-precision matching results were achieved. For oral patients, the root mean square error (RMSE) of the matching accuracy reached 0.02 to 0.04, and for craniofacial patients, the RMSE of the matching accuracy reached 0.04 to 0.06, which is much lower than the matching accuracy of the prior art (above 0.1), indicating that the matching method of the present invention can more accurately align CBCT images and STL models, thereby improving the matching accuracy and reliability.

[0069] By dynamically adjusting the function, the matching strategy is adjusted according to the patient's specific characteristics (such as age, gender, medical history, etc.), making the matching results more personalized and adaptable. This method not only improves the matching accuracy, but also better meets the needs of different patients, thereby improving the practicality and effectiveness of clinical applications.

[0070] In summary, by reasonably selecting the X-ray dose, we can ensure image quality and reduce the patient's radiation exposure risk; by selecting the appropriate scanning angle and resolution, we can cover the complete anatomical area and ensure that the details of the image are clear. The high-quality STL model provides an accurate reference for subsequent matching. Denoising and detail removal ensure the simplicity and accuracy of the model. The marking of key feature points provides important positioning information for the matching process. A large number of CBCT images and corresponding STL models are collected from multiple medical institutions for preliminary registration to form a one-to-one paired data set. The convolutional neural network is used to extract the features of the CBCT images and perform feature fusion. Its high-quality training data set provides rich samples for the training of deep learning models. The convolutional neural network can capture the complex features in the image. The feature fusion technology further enhances the representativeness of the features. The preliminary matching results provide a basis for subsequent adjustments. The dynamic adjustment strategy makes the matching results more in line with the specific conditions of the patients and improves the personalization and accuracy of the matching.

[0071] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for accurately matching CBCT data with STL data, characterized in that: The method comprises, S1. According to different application scenarios, select X-ray dose, scanning angle and resolution, and pre-process the collected CBCT images; S2, obtain the patient's STL model from the 3D scanner, remove noise and unnecessary details from the model, and identify and mark the key feature points of the model; S3, collect a large number of CBCT images and corresponding STL models, build a training data set, and use convolutional neural networks to extract features from CBCT images; S4, input the acquired CBCT images into the trained deep learning model to obtain preliminary matching results, and dynamically adjust the matching strategy according to the patient's condition; S5. Use the standard evaluation indicator root mean square error to quantitatively evaluate the matching results, and feed the evaluation results back to the deep learning model to continuously optimize the performance of the model.

2. The method for accurately matching CBCT data with STL data according to claim 1, characterized in that: In step S1, the X-ray dose, scanning angle and resolution are selected according to different application scenarios, and the steps include: S1.

1. For oral and maxillofacial surgery, use a lower dose, and for complex craniofacial structures, the dose can be appropriately increased; S1.

2. To cover the complete anatomical area, select a scanning angle between 180° and 360°. For smaller areas, select 180° scanning; for larger areas, use 360° scanning. S1.

3. For fine structures, use high resolution, and for large-scale structures, use medium resolution.

3. The method for accurately matching CBCT data with STL data according to claim 1, characterized in that: In step S1, the collected CBCT image is preprocessed, and the steps also include: S1.

4. Use adaptive filter to denoise CBCT images. The expression is: Among them, I(x,y) is the original image, I denoised (x, y) is the denoised image, σ is the standard deviation of the noise, k is the adjustment parameter, is the image gradient, is the Laplace operator; S1.5, use the histogram equalization method to enhance the contrast of the image, the expression is: I enhanced (x,y)=T(I(x,y)); Where T is the histogram equalization transformation function, I(x,y) is the denoised image, and I enhanced (x, y) is the image after contrast enhancement; S1.6, use the Canny edge detection algorithm to identify the edges in the image. The expression is: Among them, E(x,y) is the edge detection result, G x (x,y) and G y (x,y) is the derivative of the Gaussian filter, and is the gradient of the image in the x and y directions.

4. The method for accurately matching CBCT data with STL data according to claim 1, characterized in that: In step S2, the STL model of the patient is obtained from a 3D scanner, noise and unnecessary details in the model are removed, and key feature points of the model are identified and marked, and the steps include: S2.

1. Use a high-precision 3D scanner to scan the relevant parts of the patient and obtain the STL model of the patient; S2.2, use the Laplace smoothing filter to denoise the STL model, smooth the model surface and remove small-scale noise; S2.3, use curvature-based simplification algorithms to remove unnecessary details from the model; S2.

4. Use Harris corner detection algorithm to identify the key feature points of the model.

5. The method for accurately matching CBCT data with STL data according to claim 1, characterized in that: In step S3, a large number of CBCT images and corresponding STL models are collected to construct a training data set, which includes: S3.

1. Collect a large number of CBCT images from multiple medical institutions and obtain STL models corresponding to the CBCT images from a 3D scanner; S3.2, preliminarily align the collected CBCT images and STL models to ensure that they are spatially aligned; S3.

3. Align the registered CBCT image and STL model to form a one-to-one paired data set.

6. The method for accurately matching CBCT data with STL data according to claim 1, characterized in that: In step S3, a convolutional neural network is used to extract features from the CBCT image, and the steps further include: S3.

4. Design a deep convolutional neural network model to extract features of CBCT images; S3.

5. Use feature fusion technology to fuse the features of the CBCT image with the features of the STL model to form a joint feature vector, which is expressed as: F joint =α·F CBCT +β·F STL ; Among them, F joint is the joint eigenvector, F CBCT is the feature vector of CBCT image, F STL is the feature vector of the STL model, α and β are the weight coefficients of feature fusion; S3.

6. Design a composite loss function that combines the mean square error and the structural similarity index to evaluate the matching accuracy of the model. The expression is: L=λ1·MSE(F joint ,F target )+λ2·(1-SSIM(F joint ,F target )); Among them, L is the composite loss function, MSE is the mean square error, SSIM is the structural similarity index, and F joint is the joint eigenvector, F target is the target feature vector, λ1 and λ2 are the weight coefficients of the loss function.

7. The method for accurately matching CBCT data with STL data according to claim 1, characterized in that: In step S4, the collected CBCT images are input into the trained deep learning model to obtain a preliminary matching result, and the matching strategy is dynamically adjusted according to the patient's condition. The steps include: S4.

1. Input the acquired CBCT images into the trained deep learning model; S4.2, the model extracts the features of the CBCT image through a convolutional neural network and matches it with the pre-extracted STL model features to generate preliminary matching results; S4.

3. Use a dynamic adjustment function to adjust the weight of the matching strategy according to the patient characteristics. The expression is: M final =M initial +γ·f(P,M initial ); Among them, M final is the final matching result, M initial is the preliminary matching result, P is the patient feature vector, γ is the adjustment coefficient, f(P,M initial ) is a dynamic adjustment function; S4.

4. Dynamic adjustment function f(P,M initial ) Generate adjustment terms by linearly combining patient characteristics and preliminary matching results, expressed as: f(P,M initial )=w1·P1+w2·P2+…+w n ·P n +b·M initial ; Among them, P1, P2…P n are the components of the patient’s feature vector, w1, w2, ...w n is the corresponding weight, and b is the weight of the preliminary matching result.

8. The method for accurately matching CBCT data with STL data according to claim 1, characterized in that: In step S5, the matching results are quantitatively evaluated using the standard evaluation indicator root mean square error, and the evaluation results are fed back to the deep learning model to continuously optimize the performance of the model. The steps include: S5.

1. Use the root mean square error as the evaluation index to quantitatively evaluate the preliminary matching results and the final matching results. The expression is: Among them, RMSE is the root mean square error, M pred (i) is the predicted matching result, M true (i) is the actual matching result, and N is the number of matching points; S5.

2. Feed the evaluation results back to the deep learning model and update the model parameters through the back propagation algorithm; S5.

3. Use the gradient descent method to update the model parameters and reduce the root mean square error. The expression is: Among them, θ new is the updated model parameter, θ old is the current model parameter, η is the learning rate, is the gradient of the root mean square error with respect to the model parameters.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method for accurate matching of CBCT data and STL data according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for accurate matching of CBCT data and STL data according to any one of claims 1 to 8 are implemented.

Images