Oral cavity diagnosis and treatment auxiliary method and system based on augmented reality

Abstract

The invention discloses an oral cavity diagnosis and treatment auxiliary method and system based on augmented reality, and the method comprises the steps: obtaining oral cavity diagnosis and treatment operation plan auxiliary information, making a patient oral cavity AR display model through employing the obtained CT image data, and turning on an oral cavity endoscope cold light source and a high-definition camera before operation diagnosis and treatment, the position of the high-definition camera is adjusted in real time to complete collection of an operation high-definition image, the operation high-definition image is transmitted to the AR server, the AR server stores data of the whole operation process, the operation high-definition image and a patient oral cavity AR display model are fused, and the operation high-definition image is obtained. According to the AR glasses, receiving and displaying the generated operation and oral cavity AR real-time fusion image and the extracted key auxiliary information, performing further operation treatment work; the invention has the effects of improving oral disease treatment efficiency, reducing oral treatment cost and guaranteeing physical health of oral doctors.

Description

Technical Field

[0001] The present invention relates to the field of oral diagnosis and treatment assistance technology, and more specifically, to the field of oral diagnosis and treatment assistance method and system technology based on augmented reality. Background Art

[0002] As people's living standards improve, the demand for oral disease diagnosis, treatment and care is growing, and the work of dentists is becoming increasingly heavy. However, the small oral space and insufficient light make the diagnosis and treatment of oral diseases difficult and the operation difficult. Long-term abnormal standing posture causes many dentists to suffer from occupational diseases such as cervical and lumbar muscle strain, which in turn increases the cost of oral disease diagnosis and treatment.

[0003] The development and maturity of medical imaging technology has enabled the application of augmented reality (AR) technology in the medical field. AR glasses can provide real-time three-dimensional images and auxiliary information during oral surgery, such as: the patient's internal oral structure, the location of the teeth or diseased parts to be treated, the surgical plan, and commonly used imaging data, so that doctors can have a clearer understanding of the patient's oral condition during the operation, thereby improving the efficiency and accuracy of oral diagnosis and treatment, and ensuring the continuity and high precision of the operation.

[0004] Therefore, there is a need for an oral diagnosis and treatment auxiliary method and system that is efficient, low-cost, easy to operate, and can ensure the health of dentists. Summary of the invention

[0005] In view of the deficiencies in the prior art, the object of the present invention is to provide an oral diagnosis and treatment auxiliary method and system based on augmented reality.

[0006] In a first aspect, the present invention provides an augmented reality-based oral diagnosis and treatment auxiliary method, comprising the following steps:

[0007] Obtain auxiliary information on oral diagnosis and treatment surgery plans;

[0008] Acquire CT image data of the patient's oral cavity, and use the CT image data to produce an AR display model of the patient's oral cavity;

[0009] Start the surgery, turn on the cold light source of the oral endoscope and the HD camera, and adjust the HD camera until the patient's oral cavity is clearly visible;

[0010] Use high-definition cameras to collect high-definition surgical images in real time;

[0011] The AR server saves the surgical video and integrates the surgical HD image with the patient's oral AR display model to extract key auxiliary information for the surgical plan.

[0012] AR glasses wirelessly receive and display the surgical and oral AR real-time fusion images and extracted key auxiliary information in real time;

[0013] Further surgical treatment is performed based on the surgical and oral AR real-time fusion images displayed by the AR glasses and the extracted key auxiliary information.

[0014] Preferably, the oral surgery plan auxiliary information includes the equipment required for oral diagnosis and treatment surgery, basic patient information, disease diagnosis results, surgical plan, and imaging data.

[0015] Preferably, the devices required for the oral diagnosis and treatment surgery include an oral endoscope, an AR server and AR glasses.

[0016] Preferably, the oral endoscope includes a high-definition camera, a cold light source and a wireless or wired communication module; the AR server includes a data communication module, a patient oral AR display model processing module and a medical image processing module; the AR glasses include a wireless data receiving module, a real-time image decoding module and an image display module.

[0017] Preferably, obtaining CT image data of the patient's oral cavity, and using the CT image data to produce an AR display model of the patient's oral cavity, includes:

[0018] A high-resolution oral CT scanner is used to obtain CT image data of the patient's oral cavity, and the obtained CT image data is preprocessed. Based on the preprocessed CT image data, a three-dimensional model of the oral cavity is generated using a three-dimensional reconstruction algorithm. The optimized three-dimensional model of the oral cavity is integrated into the AR platform to obtain the patient's oral AR display model, and the trigger conditions and interaction methods of the patient's oral AR display model are set.

[0019] Preferably, a high-definition camera is used to collect high-definition surgical images in real time, including:

[0020] The doctor performs surgical treatment on the oral diseased part and ensures the real-time acquisition of high-definition surgical images by timely adjusting the lens position and angle of the high-definition camera. The acquired high-definition surgical images are then transmitted to the AR server by wire or wireless means.

[0021] Preferably, the AR server saves the surgical video and fuses the surgical high-definition image with the patient's oral AR display model to extract key auxiliary information of the surgical plan, including:

[0022] After the AR server receives the real-time high-definition surgical images via wired or wireless means, it saves them in video format and uses a feature-based fusion algorithm to fuse the surgical high-definition images with the pre-generated patient oral AR display model. It uses an image processing algorithm to analyze the surgical video and high-definition images, extracts key auxiliary information, optimizes surgical operations, and transmits the real-time generated surgical and oral AR real-time fusion images and the extracted key auxiliary information to the AR glasses via wireless means.

[0023] Preferably, the AR glasses wirelessly receive and display in real time the surgical and oral AR real-time fusion images and the extracted key auxiliary information, including:

[0024] The real-time collected surgical and oral AR real-time fusion images and the extracted key auxiliary information are transmitted to the AR glasses wirelessly. During the transmission process, the AR glasses use image compression and encryption technology to ensure the integrity and security of the surgical and oral AR real-time fusion images and the extracted key auxiliary information. The AR glasses decode and render the received surgical and oral AR real-time fusion images and the extracted key auxiliary information. The AR glasses display the processed surgical and oral AR real-time fusion images in the doctor's field of view in real time and superimpose the extracted key auxiliary information on the generated virtual image.

[0025] In the second aspect, an augmented reality-based oral diagnosis and treatment auxiliary system includes: a surgical plan auxiliary information management module, a CT image data processing and AR modeling module, a surgical equipment control module, a real-time image acquisition and processing module, an AR fusion and video storage module, an AR display and interaction module, and a data storage and backup module;

[0026] Surgical plan auxiliary information management module: responsible for acquiring, storing and managing oral diagnosis and treatment surgical plan auxiliary information;

[0027] CT image data processing and AR modeling module: obtain the CT image data of the patient's oral cavity and build an AR display model of the patient's oral cavity;

[0028] Surgical equipment control module: controls the switch of the oral endoscope, cold light source and high-definition camera, and adjusts parameters such as brightness and focal length;

[0029] Real-time image acquisition and processing module: uses a high-definition camera to capture high-definition images during surgery in real time and pre-processes the acquired high-definition images;

[0030] AR fusion and video storage module: The AR server stores the surgical video and integrates the surgical high-definition image and the patient's oral AR display model;

[0031] AR display and interaction module: The fused surgical and oral AR real-time fusion images and key auxiliary information are transmitted to the AR glasses wirelessly, and the doctor performs the surgery according to the display of the AR glasses;

[0032] Data storage and backup module: Use the AR server to store and back up the large amount of video, audio, and image data recorded by the AR glasses during the entire surgical process.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] The device and system provided by the present invention allow dentists to work for a long time in a healthy posture, free from occupational diseases such as cervical vertebrae and lumbar vertebrae;

[0035] The oral endoscope provided by the present invention enables dentists to clearly see the inside of the patient's oral cavity and the condition of the affected area at any time, thereby improving the efficiency and accuracy of oral disease treatment and reducing treatment costs;

[0036] The present invention simplifies the functions of AR glasses, and only requires AR glasses to have real-time image decoding and display functions, so the cost is low, the power consumption is low, and the weight is light, so that dentists can wear AR glasses to work for a long time;

[0037] The method provided by the present invention will use video to record the entire oral surgery process, which is helpful for surgery review and teaching, and can also be used as evidence to avoid medical disputes when necessary. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A flowchart of an augmented reality-based oral diagnosis and treatment auxiliary method is proposed for the present invention;

[0039] Figure 2 The present invention proposes a flow chart of an oral diagnosis and treatment auxiliary system based on augmented reality. DETAILED DESCRIPTION

[0040] Reference Figure 1 to Figure 2 .

[0041] The embodiments further illustrate an augmented reality-based oral diagnosis and treatment auxiliary method and system proposed by the present invention.

[0042] Reference Figure 1 , an oral diagnosis and treatment auxiliary method based on augmented reality, comprising the following steps:

[0043] Obtain auxiliary information on oral diagnosis and treatment surgery plans;

[0044] Acquire CT image data of the patient's oral cavity, and use the CT image data to produce an AR display model of the patient's oral cavity;

[0045] Start the surgery, turn on the cold light source of the oral endoscope and the HD camera, and adjust the HD camera until the patient's oral cavity is clearly visible;

[0046] Use high-definition cameras to collect high-definition surgical images in real time;

[0047] The AR server saves the surgical video and integrates the surgical HD image with the patient's oral AR display model to extract key auxiliary information for the surgical plan.

[0048] AR glasses wirelessly receive and display the surgical and oral AR real-time fusion images and extracted key auxiliary information in real time;

[0049] Further surgical treatment is performed based on the surgical and oral AR real-time fusion images displayed by the AR glasses and the extracted key auxiliary information.

[0050] The oral surgery plan auxiliary information includes the equipment required for oral diagnosis and treatment surgery, basic patient information, disease diagnosis results, surgical plan, and imaging data.

[0051] The patient's basic information includes the location of the tooth or affected area to be treated, the patient's previous history of oral disease, systemic disease history, surgical history and drug allergy history; the imaging data includes oral X-rays, oral CT, oral MRI, ultrasound and radionuclide examination.

[0052] The devices required for the oral diagnosis and treatment surgery include an oral endoscope, an AR server and AR glasses.

[0053] The oral endoscope includes a high-definition camera, a cold light source, and a wireless or wired communication module; the AR server includes a data communication module, a patient oral AR display model processing module, and a medical image processing module; the AR glasses include a wireless data receiving module, a real-time image decoding module, and an image display module;

[0054] The AR server has a system platform inside, which receives and saves real-time compressed images from the oral endoscope. It can also be integrated with the patient's oral AR display model data generated in the early stage and transmitted to the AR glasses via wireless. The real-time images from the oral endoscope are integrated and analyzed by the AR server to generate high-definition complete surgical playback, which can be used for postoperative analysis and teaching, and can also be used for surgical quality evaluation, while providing key evidence to avoid medical disputes.

[0055] AR glasses receive surgical and oral AR real-time fusion images in real time through wireless modules, and display them in real time on AR glasses after being decoded by decoding modules.

[0056] Obtain CT image data of the patient's oral cavity, and use the CT image data to produce an AR display model of the patient's oral cavity, including:

[0057] A high-resolution oral CT scanner is used to obtain CT image data of the patient's oral cavity, and the obtained CT image data is preprocessed using image processing software. Based on the preprocessed CT image data, a three-dimensional model of the oral cavity is generated using a three-dimensional reconstruction algorithm, and the optimized three-dimensional model of the oral cavity is integrated into the AR platform to obtain an AR display model of the patient's oral cavity, and the trigger conditions and interaction methods of the AR display model of the patient's oral cavity are set;

[0058] The preprocessing of the acquired CT image data includes: data loading and format conversion, noise suppression, window width and window position adjustment, normalization, image segmentation, three-dimensional reconstruction, and quality control.

[0059] Use high-definition cameras to collect high-definition surgical images in real time, including:

[0060] The doctor performs surgical treatment on the oral diseased part and ensures the real-time acquisition of high-definition surgical images by timely adjusting the lens position and angle of the high-definition camera. The acquired high-definition surgical images are then transmitted to the AR server by wire or wireless means.

[0061] The AR server saves the surgical video and integrates the surgical HD image with the patient's oral AR display model to extract key auxiliary information of the surgical plan, including:

[0062] After the AR server receives the real-time high-definition surgical images via wired or wireless means, it saves them in video format and uses a feature-based fusion algorithm to fuse the surgical high-definition images with the pre-generated patient oral AR display model. It uses an image processing algorithm to analyze the surgical video and high-definition images, extracts key auxiliary information, optimizes surgical operations, and transmits the real-time generated surgical and oral AR real-time fusion images and the extracted key auxiliary information to the AR glasses via wireless means.

[0063] AR glasses wirelessly receive and display surgical and oral AR real-time fusion images and extracted key auxiliary information in real time, including:

[0064] The real-time collected surgical and oral AR real-time fusion images and the extracted key auxiliary information are transmitted to the AR glasses wirelessly. During the transmission process, the AR glasses use image compression and encryption technology to ensure the integrity and security of the surgical and oral AR real-time fusion images and the extracted key auxiliary information. The AR glasses decode and render the received surgical and oral AR real-time fusion images and the extracted key auxiliary information. The AR glasses display the processed surgical and oral AR real-time fusion images in the doctor's field of view in real time and superimpose the extracted key auxiliary information on the generated virtual image.

[0065] Reference Figure 2 , an oral diagnosis and treatment auxiliary system based on augmented reality, including: surgical plan auxiliary information management module, CT image data processing and AR modeling module, surgical equipment control module, real-time image acquisition and processing module, AR fusion and video storage module, AR display and interaction module, data storage and backup module;

[0066] Surgical plan auxiliary information management module: responsible for acquiring, storing and managing oral diagnosis and treatment surgical plan auxiliary information;

[0067] CT image data processing and AR modeling module: obtain the CT image data of the patient's oral cavity and build an AR display model of the patient's oral cavity;

[0068] Surgical equipment control module: controls the switch of the oral endoscope, cold light source and high-definition camera, and adjusts parameters such as brightness and focal length;

[0069] Real-time image acquisition and processing module: uses a high-definition camera to capture high-definition images during surgery in real time and pre-processes the acquired high-definition images;

[0070] AR fusion and video storage module: The AR server stores the surgical video and integrates the surgical high-definition image and the patient's oral AR display model;

[0071] AR display and interaction module: The fused surgical and oral AR real-time fusion images and key auxiliary information are transmitted to the AR glasses wirelessly, and the doctor performs the surgery according to the display of the AR glasses;

[0072] Data storage and backup module: Use the AR server to store and back up the large amount of video, audio, and image data recorded by the AR glasses during the entire surgical process.

[0073] Other uses of the present invention: For medical teaching, real-time images are uploaded to the AR server through the high-definition camera of the oral endoscope. The recognition function of the AR server can be used to analyze the video and accurately identify the various tissue structures in the oral cavity, and relevant information, surgical methods and other medical knowledge can be displayed on the AR glasses.

[0074] Possible alternatives of the present invention:

[0075] The present invention can be applied without a cold light source;

[0076] The present invention can be applied without oral AR display model data;

[0077] The present invention can be applied without auxiliary information of surgical plan.

[0078] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. An oral diagnosis and treatment auxiliary method based on augmented reality, characterized in that: The following steps are involved: Obtain auxiliary information on oral diagnosis and treatment surgery plans; Acquire CT image data of the patient's oral cavity, and use the CT image data to produce an AR display model of the patient's oral cavity; Start the surgery, turn on the cold light source of the oral endoscope and the HD camera, and adjust the HD camera until the patient's oral cavity is clearly visible; Use high-definition cameras to collect high-definition surgical images in real time; The AR server saves the surgical video and integrates the surgical HD image with the patient's oral AR display model to extract key auxiliary information for the surgical plan. AR glasses wirelessly receive and display the surgical and oral AR real-time fusion images and extracted key auxiliary information in real time; Further surgical treatment is performed based on the surgical and oral AR real-time fusion images displayed by the AR glasses and the extracted key auxiliary information.

2. The oral diagnosis and treatment auxiliary method based on augmented reality according to claim 1, characterized in that: The oral surgery plan auxiliary information includes the equipment required for oral diagnosis and treatment surgery, basic patient information, disease diagnosis results, surgical plan, and imaging data.

3. The oral diagnosis and treatment auxiliary method based on augmented reality according to claim 2, characterized in that: The devices required for the oral diagnosis and treatment surgery include an oral endoscope, an AR server and AR glasses.

4. The oral diagnosis and treatment auxiliary method based on augmented reality according to claim 3, characterized in that: The oral endoscope includes a high-definition camera, a cold light source and a wireless or wired communication module; the AR server includes a data communication module, a patient oral AR display model processing module and a medical image processing module; the AR glasses include a wireless data receiving module, a real-time image decoding module and an image display module.

5. The oral diagnosis and treatment auxiliary method based on augmented reality according to claim 4, characterized in that: Obtain CT image data of the patient's oral cavity, and use the CT image data to produce an AR display model of the patient's oral cavity, including: Use a high-resolution oral CT scanner to obtain CT image data of the patient's oral cavity, preprocess the acquired CT image data, generate a three-dimensional model of the oral cavity based on the preprocessed CT image data using a three-dimensional reconstruction algorithm, integrate the optimized three-dimensional model of the oral cavity into the AR platform, obtain the patient's oral AR display model, and set the trigger conditions and interaction methods of the patient's oral AR display model.

6. The oral diagnosis and treatment auxiliary method based on augmented reality according to claim 5, characterized in that: Use high-definition cameras to collect high-definition surgical images in real time, including: The doctor performs surgical treatment on the oral diseased part and ensures the real-time acquisition of high-definition surgical images by timely adjusting the lens position and angle of the high-definition camera. The acquired high-definition surgical images are then transmitted to the AR server by wire or wireless means.

7. The oral diagnosis and treatment auxiliary method based on augmented reality according to claim 6, characterized in that: The AR server saves the surgical video and integrates the surgical HD image with the patient's oral AR display model to extract key auxiliary information of the surgical plan, including: After the AR server receives the real-time high-definition surgical images via wired or wireless means, it saves them in video format and uses a feature-based fusion algorithm to fuse the surgical high-definition images with the pre-generated patient oral AR display model. It uses an image processing algorithm to analyze the surgical video and high-definition images, extracts key auxiliary information, optimizes surgical operations, and transmits the real-time generated surgical and oral AR real-time fusion images and the extracted key auxiliary information to the AR glasses via wireless means.

8. The oral diagnosis and treatment auxiliary method based on augmented reality according to claim 7, characterized in that: AR glasses wirelessly receive and display surgical and oral AR real-time fusion images and extracted key auxiliary information in real time, including: The real-time collected surgical and oral AR real-time fusion images and the extracted key auxiliary information are transmitted to the AR glasses wirelessly. During the transmission process, the AR glasses use image compression and encryption technology to ensure the integrity and security of the surgical and oral AR real-time fusion images and the extracted key auxiliary information. The AR glasses decode and render the received surgical and oral AR real-time fusion images and the extracted key auxiliary information. The AR glasses display the processed surgical and oral AR real-time fusion images in the doctor's field of view in real time and superimpose the extracted key auxiliary information on the generated virtual image.

9. An oral diagnosis and treatment auxiliary system based on augmented reality, characterized in that: The system is used to implement an augmented reality-based oral diagnosis and treatment auxiliary method as described in any one of claims 1 to 8, comprising: a surgical plan auxiliary information management module, a CT image data processing and AR modeling module, a surgical equipment control module, a real-time image acquisition and processing module, an AR fusion and video storage module, an AR display and interaction module, and a data storage and backup module; Surgical plan auxiliary information management module: responsible for acquiring, storing and managing oral diagnosis and treatment surgical plan auxiliary information; CT image data processing and AR modeling module: obtain the CT image data of the patient's oral cavity and build an AR display model of the patient's oral cavity; Surgical equipment control module: controls the switch of the oral endoscope, cold light source and high-definition camera, and adjusts parameters such as brightness and focal length; Real-time image acquisition and processing module: uses a high-definition camera to capture high-definition images during surgery in real time and pre-processes the acquired high-definition images; AR fusion and video storage module: The AR server stores the surgical video and integrates the surgical high-definition image and the patient's oral AR display model; AR display and interaction module: The fused surgical and oral AR real-time fusion images and key auxiliary information are transmitted to the AR glasses wirelessly, and the doctor performs the surgery according to the display of the AR glasses; Data storage and backup module: Use the AR server to store and back up the large amount of video, audio, and image data recorded by the AR glasses during the entire surgical process.

Images