Oral multi-specialty endoscope and image artificial intelligence processing assisted diagnosis method

Abstract

The present invention relates to the technical field of oral treatment instruments, and provides an oral multi-specialty endoscope and an image artificial intelligence processing assisted diagnosis method. Among them, the oral multi-specialty endoscope includes: an insertion part, an operating part, a signal line and an optical fiber; the front end of the operating part is connected to the insertion part, and the front end of the insertion part is provided with an image acquisition element, the image acquisition element is connected to the signal line, and the signal line is connected to the host; a light source is provided inside the operating part or the host, and when the light source is located in the host, the light is transmitted to the insertion part through the optical fiber; the shape of the mirror tube of the insertion part is a hook shape, a bend shape, a straight rod shape, or other observable shapes. The embodiment of the present invention can directly examine the fine structure of oral tissues such as tooth pulp cavity root canal, periodontal, implant cavity and salivary duct.

Description

Technical Field

[0001] The present invention relates to the technical field of oral treatment instruments, and in particular to an oral multi-specialty endoscope and an image artificial intelligence processing auxiliary diagnosis method. Background Art

[0002] The oral cavity is part of the whole body. Oral diseases not only affect the function of the oral organs, but also can cause or aggravate the lesions of other organs in the whole body, causing a significant impact on the overall health. Dental caries and periodontal diseases are the two most common diseases that endanger the oral health of Chinese residents. They not only destroy the hard tissues of the teeth and the supporting tissues around the teeth, affecting the functions of chewing, speech, and aesthetics, but some pathogenic microorganisms exist in the oral cavity for a long time, which can cause or aggravate certain systemic diseases such as coronary heart disease and diabetes, endangering the health of the whole body.

[0003] Identifying oral diseases, making clear diagnoses, and implementing precise treatments are particularly important in oral clinical work. Oral examination mirrors are oral examination instruments that were adopted earlier and are frequently used in daily life. They use the reflected light from the glass mirror to illuminate and increase the field of view. Inspection mirrors use a simple glass mirror as an observation tool with a simple structure, and cannot view oral tissues and internal fine structures. Dental surgical microscopes were used in oral clinics internationally in the 1990s and are now widely used in oral medicine. Surgical microscopes are used to provide lighting, magnify the field of view, and improve the refinement of oral treatment operations. However, dental surgical microscopes also have shortcomings in use:

[0004] 1. It must be imaged through a special microscopic mouth mirror. As a medical consumable, the microscopic mouth mirror increases medical expenses and has a limited operating area. The rectilinear propagation of light causes some places that cannot be observed by the microscope, such as the curved root canal and the palatal side of the root apex during apical surgery. The doctor cannot use the microscope to observe its delicate structure.

[0005] 2. The operation time is long. When observing subtle lesions, the magnification and light intensity need to be increased, which can easily lead to eye fatigue, dizziness, nausea and other symptoms.

[0006] 3. The patient opens his mouth widely and has poor cooperation, making it difficult for him to maintain a stable position. Once the body position is changed, the microscope imaging field of view of the lesion will become blurred or disappear, and the physician will have to readjust it, wasting time and reducing operation efficiency and treatment effect.

[0007] The oral endoscope is an important tool for oral examination, but the existing oral endoscope probe is large, has limited focus, and low imaging clarity. It is still at the primary medical device stage of oral digital observer and cannot meet the needs of dental pulp root canal, periodontal pocket, implant cavity and salivary gland examination and treatment. It cannot be called a medical-grade oral endoscope in the strict sense.

[0008] The oral multi-specialty endoscope of the present invention is a miniature camera system that places an image acquisition element in the oral cavity. The system stores the signal in a computer, and can conveniently change the focal length and camera range as needed, so as to clearly observe the fine structure of oral tissues such as tooth pulp root canals, periodontal pockets, implant cavities and salivary glands, detect oral tissue lesions, and take timely treatment measures. The present invention solves a problem that needs to be solved urgently in clinical practice, and the oral multi-specialty endoscope is a further extension of the dentist's eyes. The oral cavity has a complex structure, many types of diseases, and complicated conditions. The existing oral endoscopes have insufficient clinical value and cannot meet the needs of clinical examination and treatment. In view of this, the present invention is proposed. Summary of the invention

[0009] The purpose of the present invention is to provide an oral multi-specialty endoscope and an image artificial intelligence processing auxiliary diagnosis method to solve the above problems.

[0010] On the one hand, an embodiment of the present invention provides an oral multi-specialty endoscope, comprising: an insertion portion, an operation portion, an operation button, an image acquisition element, a signal line, and an optical fiber;

[0011] The front end of the operating part is connected to the insertion part, the front end of the insertion part is provided with an image acquisition element, the image acquisition element is connected to the signal line, and the signal line is connected to the host;

[0012] A light source is disposed inside the operating portion or the main unit. When the light source is located in the main unit, light is transmitted to the insertion portion through the optical fiber.

[0013] The mirror tube of the insertion portion is in the shape of a hook, a bend, a straight rod, or other shapes that can be observed.

[0014] On the other hand, an embodiment of the present invention provides an image artificial intelligence processing assisted diagnosis method, based on the above-mentioned oral multi-specialty endoscope, the method includes:

[0015] Obtaining an image of the patient's oral cavity;

[0016] The oral images are processed using AI algorithm models and medical image algorithms to automatically identify oral diseases, including dental pulp, root canal, mucosal, periodontal, salivary gland, and implant cavity diseases.

[0017] On the other hand, an embodiment of the present invention provides an image artificial intelligence processing method, based on the above-mentioned oral multi-specialty endoscope, the method includes:

[0018] Acquire an image of the treatment area before treatment as a standard image;

[0019] After treatment, adjusting light source parameters, and taking images of the treatment area under different parameters as a plurality of images to be matched;

[0020] Calculating the histogram matching degree between each to-be-matched image and the standard image;

[0021] The image to be matched with the highest matching degree is selected, and histogram matching is performed with the standard image. The matched image is used for comparison with the standard image and / or for displaying the treatment effect.

[0022] The beneficial effects of the embodiments of the present invention are:

[0023] 1. In endodontic root canal treatment, the endoscope image acquisition element can directly observe the fine structure of the endodontic root canal, find the pits and fissures, find the caries site, determine the range of caries tissue, find the pulp foramen, assist in removing the top and cleaning the pulp cavity calcification during pulp opening, find the root canal opening, and observe the state of the pulp and root canal. In apical surgery, the location and range of lesions of the soft and hard tissues outside the apex can be explored to meet the clinical needs of oral disease examination and treatment.

[0024] 2. In periodontal treatment, this endoscope can be used to clearly detect periodontitis, tartar conditions in the root and root bifurcation area, abnormal conditions such as cracks, depressions, hyperplasia and external absorption on the outer surface of the root, and can also be used for flushing, medication and other treatments.

[0025] 3. In dental implants, for those who need to lift the maxillary sinus to complete maxillary dental implants, this endoscope can penetrate deep into the implant cavity, detect the condition of the maxillary sinus floor mucosa, safely and efficiently complete the maxillary sinus lifting surgery, and assist in the treatment of peri-implantitis.

[0026] 4. When there is obstruction and inflammatory diseases in the oral salivary glands, this endoscope can penetrate deep into the salivary ducts to explore the location of the lesion structure, and can also be used for flushing, medication and other treatments.

[0027] 5. The embodiments of the present invention can utilize artificial intelligence technology to automatically identify oral diseases based on oral images, thereby providing auxiliary diagnosis and intelligent diagnosis functions. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0029] Figure 1 A schematic diagram of the overall structure of an oral multi-specialty endoscope provided in one embodiment of the present invention.

[0030] Figure 2 for Figure 1 Schematic diagram of the structure of the front end of the middle insertion part.

[0031] Figure 3 A schematic diagram of the shape of the insertion tube provided in one embodiment of the present invention.

[0032] Figure 4 A schematic diagram of another shape of an insertion tube provided in an embodiment of the present invention.

[0033] Figure 5 A schematic diagram of another shape of an insertion tube provided in accordance with an embodiment of the present invention.

[0034] Figure 6 A schematic diagram of the overall structure of another oral multi-specialty endoscope provided in one embodiment of the present invention.

[0035] Figure 7 for Figure 6 Schematic diagram of the structure of the front end of the middle insertion part.

[0036] Figure 8 It is a flow chart of an image artificial intelligence processing assisted diagnosis method provided by an embodiment of the present invention.

[0037] Fig. 9 It is a flow chart of an image artificial intelligence processing method provided by an embodiment of the present invention.

[0038] Among them, the accompanying drawings are marked as follows:

[0039] 1-insertion part, 11-image acquisition element, 12-optical fiber, 13-mirror tube;

[0040] 2- Operation part;

[0041] 3- Operation button;

[0042] 4-Signal line;

[0043] 5- Jacket;

[0044] 6-Multi-purpose hollow channel. DETAILED DESCRIPTION

[0045] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0046] It should be noted that when a component is referred to as being "fixed on" or "disposed on" another component, it may be directly or indirectly located on the other component. When a component is referred to as being "connected to" another component, it may be directly or indirectly connected to the other component. The directions or positions indicated by the terms "upper", "lower", "left", "right", "front", "back", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc. are based on the directions or positions shown in the accompanying drawings and are only for the convenience of description and cannot be understood as limitations on the present technical solution. The terms "first" and "second" are only used for the convenience of description and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features. "Multiple" means two or more, and "several" means any number including one, unless otherwise clearly and specifically defined.

[0047] Combination Figure 1 and Figure 2 , the embodiment of the present application provides an oral multi-specialty endoscope, which can realize oral multi-specialty inspection and treatment, and can be used for inspection and treatment of dental pulp root canals, periodontal pockets, implant cavities and salivary glands. The oral multi-specialty endoscope includes: an insertion part 1, an operating part 2, a signal line 4 and an optical fiber 12; the front end of the operating part 2 is connected to the insertion part 1, and an image acquisition element 11 is arranged at the front end of the insertion part 1 or inside the operating part 2, and the image acquisition element 11 is connected to the signal line 4, and the signal line 4 is connected to a host (not shown in the figure); a light source is arranged inside the operating part 2 or the host, and light is transmitted to the insertion part 1 through the optical fiber 12; in order to meet the different needs of oral clinical multi-specialties, the mirror tube 13 of the insertion part 1 is in the shape of a hook, a bend, a straight rod, or other shapes that can be observed. The endoscope can directly inspect various fine structures such as dental pulp root canals, periodontal, implant cavities and salivary ducts.

[0048] Specifically, Figure 3 A hook-type insertion tube shape is shown, and the hook shape is like a probe; Figure 4 A bent insertion tube shape is shown, with a bending angle between 60° and 150°; Figure 5 A straight-rod-type insertion tube shape is shown.

[0049] Optionally, the oral multi-specialty endoscope is made of hard material, and a jacket 5 can be added to the outside of the insertion portion 1, and the jacket 5 is made of flexible material. In clinical practice, the jacket 5 can prevent cross infection.

[0050] Optionally, an operation button 3 is provided on the operation portion 2, and the operation button 3 is used to adjust light source parameters and control shooting operations.

[0051] Optionally, an image processing unit (not shown in the figure) is added to the operating part 2 according to clinical needs, and the image processing unit is used to implement image processing.

[0052] Optionally, according to clinical needs, the image operation buttons on the operating part 2 can be placed on the host or in a foot-operated mode.

[0053] Optionally, the image acquisition element 11 may be a small photographing element, which is disposed at the front end of the insertion portion 1 , and the small photographing element may be a small CCD element or a small CMOS element.

[0054] Optional, combined Figure 6 and Figure 7 The operating part 2 may also have a multi-purpose hollow channel 6, which runs from the operating part 2 to the insertion part 1; the multi-purpose hollow channel 6 is used to inject flushing liquid and / or medicine into the oral tissue, or to treat the affected part with a laser optical fiber through the channel.

[0055] The oral multi-specialty endoscope of this embodiment uses a micro image acquisition element 11 as an image receiving device; the image acquisition element 11 and the optical fiber 12 are encapsulated in the insertion part 1, and the image acquisition element 11 is connected to the image processing system or the host through the signal line 4, and the image is obtained on the monitor. The operation part 2 is equipped with an operation button 3, and the doctor can operate the button 3 to capture the patient's part and retain the image data.

[0056] In summary, the present embodiment provides an oral multi-specialty endoscope, which is an ultra-clear and ultra-fine endoscope for oral multi-specialty inspection and treatment developed based on many years of clinical work experience. It is a miniature camera system that can store signals in a computer and can change the focal length and camera range as needed; it can be used for inspection and treatment of oral tissues such as dental pulp root canals, periodontal pockets, implant cavities, salivary glands and mucosa according to clinical needs, and it also has image AI processing and auxiliary diagnosis functions. Through the endoscope, you can clearly see the fine structures of oral tissues such as the inside of the mouth, the surface of the teeth, the root canals of the dental pulp cavity, the periodontal pockets, the implant cavities, the salivary glands and mucosa. It is a further extension of the dentist's eyes. At the same time, the endoscope has image AI processing, auxiliary diagnosis, and intelligent diagnosis functions, which can effectively reduce the burden on doctors and assist doctors in accurate diagnosis and treatment. Specifically, the oral multi-specialty endoscope can achieve the following beneficial effects in different oral specialties:

[0057] 1. When the endoscope of this embodiment is used for dental pulp and root canal inspection and treatment, the endoscope can be used to directly observe various pits and fissures on the tooth, the scope of carious tissue, the pulp foramen, and assist in removing the top to clean the pulp cavity calcification during pulp opening, find the root canal opening, and observe the root canal morphology and pulp status below the root canal opening. During apical surgery, the scope of lesions of soft and hard tissues outside the apex is explored to meet the medical application of various diseases in the treatment of dental pulp. With the help of clear imaging of the endoscope, the doctor can accurately diagnose the patient's disease type, accurately distinguish between caries and pulp disease, and accurately judge the following conditions: the degree of removal of carious tissue, the exposure of the pulp, the residual calcification in the pulp cavity, the position of the root canal opening, and the residual inflammatory tissue around the root after root cutting during apical surgery. The doctor uses the endoscope to make a clear and accurate diagnosis, and uses the endoscope to perform precise operations to achieve successful treatment results.

[0058] 2. When the endoscope of this embodiment is used for periodontal examination and treatment, the patient's periodontal lesions can be clearly observed through the endoscope, and targeted treatment can be carried out. With the help of an endoscope, the periodontal pocket can be entered to clearly explore the tartar condition of the root and the root bifurcation area, and the abnormalities such as cracks, depressions, hyperplasia and external absorption on the outer surface of the root. Using the multi-purpose hollow channel of the endoscope, it is possible to inject flushing fluid and medicine into the periodontal pocket, or introduce laser optical fiber for precise treatment. The doctor can use an endoscope, a scraper, a periodontal scraper and other instruments to clean and scrape the patient's periodontium, remove tartar and plaque in the periodontal pocket, and treat periodontal inflammation. The doctor uses an endoscope to perform surgical treatments such as flap surgery, bone grafting, and guided tissue regeneration on the patient's periodontium to help the patient's periodontium recover health. The doctor can evaluate the effect of periodontal treatment with the help of an endoscope. Only thorough treatment can ensure good results of periodontal treatment.

[0059] 3. During dental implant surgery, the endoscope can penetrate deep into the bottom of the implant cavity, explore the deep situation, and determine whether nerves and blood vessels are touched; for those who need to make a window through the alveolar ridge to lift the maxillary sinus to complete the maxillary dental implant, the endoscope head can penetrate deep into the implant cavity, directly observe the condition of the maxillary sinus floor mucosa, accurately lift the sinus floor mucosa, avoid iatrogenic perforation of the maxillary sinus floor, and efficiently complete the implant surgery.

[0060] 4. During the examination and treatment of oral salivary glands, when there are obstructions and inflammatory diseases in the salivary glands, this endoscope can penetrate deep into the salivary ducts to explore the location and status of the lesions. The multi-purpose hollow channel can be used to flush, administer drugs, and aspirate the salivary ducts, effectively preventing lateral puncture of the ducts, accurately treating and unblocking the ducts, and effectively treating inflammation of the glands. The endoscope is combined with surgical instruments such as baskets, gripping forceps, and laser optical fibers to perform lithotripsy, stone basket grabbing, and negative pressure aspiration to remove stones from the salivary ducts. This endoscope has an image AI processing-assisted diagnosis function, which can assist in the diagnosis of diseases such as duct stones, deformities, and inflammation.

[0061] Based on the above oral multi-specialty endoscopes, Figure 8 This is a flow chart of an image artificial intelligence processing assisted diagnosis method provided by an embodiment of the present invention. This method can be executed by the AI ​​assisted diagnosis module in the endoscope, or by other image processing equipment, to play the role of auxiliary diagnosis and intelligent diagnosis. Figure 8 As shown, the method comprises the following steps:

[0062] S110, obtaining an oral cavity image of the patient.

[0063] After receiving the oral cavity image of the patient, the image is first preprocessed and enhanced to improve the quality of the oral cavity image. Optionally, the oral cavity image may be subjected to operations such as denoising, smoothing, and contrast enhancement.

[0064] S120. Process the oral image using an AI algorithm model and a medical image algorithm to automatically identify oral diseases, including diseases of the dental pulp, root canal, mucosa, salivary glands, periodontium, and implant cavities.

[0065] This embodiment adopts a lightweight AI algorithm recognition model obtained by knowledge distillation and transfer learning methods. Its input is a preprocessed oral image, and its output is the oral tissue type, lesion area box diagram and disease type in the image. For example, the recognition result of a caries image is: the lesion location marked by a box diagram in the image, and the note next to the box diagram: caries.

[0066] In a specific embodiment, the training method of the model includes:

[0067] S1: Data preparation: Collect oral image data and construct an oral image dataset.

[0068] S1-1: The oral image dataset contains multiple oral tissue types, including multiple pictures. Professional dentists annotate each oral picture with its real oral tissue type and disease type, and each picture corresponds to two labels.

[0069] S1-2: Data preprocessing and expansion: Normalize the oral image constructed in step S1 and transform the original data into a corresponding standard form.

[0070] S1-3: Divide the data set: Divide the image data set into two parts: training set and validation set. The training set is used to train the network model, and the validation set is used to verify the model recognition effect.

[0071] S2: Determine the teacher model and the student model. Optionally, the backbone network of the teacher model can select a YOLOv8 model with higher accuracy, better stability and larger model, and the student model can select a YOLO_Nano network with a smaller model.

[0072] S3: Use transfer learning methods to train the teacher model to obtain a suitable teacher model. Optionally, the teacher model first loads the pre-trained weights on a large public dataset, retains the network structure before the last layer, and adds a multi-layer perceptron after the network structure, so that the final output of the multi-layer perceptron includes two classification labels of oral tissue type and disease type. Then, the oral image dataset constructed in S1 is used to train the multi-layer perceptron to obtain a trained teacher model.

[0073] S4: The student model is initialized. The transfer learning method is also used to load the pre-trained weights on a large public dataset, retain the network structure before the last layer, and add a multi-layer perceptron after the network structure, so that the final output of the multi-layer perceptron includes two classification labels: oral tissue type and disease type.

[0074] S5: Use the knowledge distillation method to use the trained teacher model to guide the student model and obtain the optimal student model.

[0075] S6: Input the images in the validation set into the student model for testing, and obtain the recognition results through forward propagation calculation. The trained student model can be integrated into chips and small devices, making it convenient for the devices to be used anytime and anywhere, especially when the AI-assisted diagnosis module is deployed in mobile devices and small devices.

[0076] In one embodiment, in combination Fig. 9 In some scenarios where it is necessary to compare images before and after treatment to evaluate the treatment effect, this embodiment also provides the following image artificial intelligence processing method, which implements the treatment effect comparison or display function through the following steps:

[0077] S210, obtaining an image of the treatment area before treatment as a standard image.

[0078] The treatment area refers to the oral area of ​​the patient to be treated; specifically for dental treatment, the treatment area may refer to the patient's tooth surface pits, pulp cavity and other treatment sites. Before treatment, the dentist can adjust the light source parameters through the image system and take an image with better visual effects under a certain parameter combination as a standard image. The parameters include at least one of the power, brightness and color temperature of the light source. When only one adjustable parameter is included, each value of the single parameter is also regarded as a parameter combination.

[0079] S220 , adjusting light source parameters after treatment, and taking images of the treatment area under different parameters as a plurality of images to be matched.

[0080] After treatment, the light source parameters can be further adjusted, and a post-treatment image can be taken under different parameter combinations as multiple images to be matched.

[0081] S230: Calculate the histogram matching degree between each to-be-matched image and the standard image.

[0082] For each image to be matched, the histogram matching degree between the image and the standard image is calculated. The matching degree is used to measure the degree of visual similarity between the image to be matched and the standard image. Optionally, the two images can be converted into grayscale images and the histogram matching degree of the two grayscale images can be calculated; or the histogram matching degree of the two images in the red, green and blue channels can be calculated and the three histogram matching degrees can be averaged as the final histogram matching degree.

[0083] Specifically, the histogram matching degree can be represented by the correlation of histogram data, chi-square test, intersection test or Bhattacharyya distance. The specific representation method is the existing technology. Taking the correlation as an example, the two histograms can be recorded as H1 and H2 respectively, and the histogram matching degree d(H1, H2) of the two images is:

[0084]

[0085] in,

[0086]

[0087] Among them, u = 1 or 2, which is used to represent the index of the histogram; N represents the number of grayscale partitions of each histogram, I and J both represent the grayscale partition index, and H u (I) and H u (J) represent the values ​​corresponding to the grayscale partitions I and J in the histogram with index u respectively.

[0088] S240, selecting an image to be matched with the highest histogram matching degree, and performing histogram matching with the standard image. The matched image is used for comparison with the standard image and / or for displaying the treatment effect.

[0089] After the histogram matching degree of each image to be matched and the standard image is calculated, an image to be matched with the highest histogram matching degree is selected. The histogram of this image is similar to that of the standard image. Using this image for histogram matching with the standard image can reduce the large adjustment of the histogram, thereby reducing the detail distortion caused by histogram matching.

[0090] Specifically, the role of histogram matching is to make the histogram distribution of one image basically consistent with that of another image, so that the two images maintain consistent visual effects, such as basically consistent tones, which makes it easier to compare treatment effects. The basic principles of histogram matching are as follows:

[0091] First, the image to be matched and the standard image are converted into grayscale images, which are called the grayscale image to be matched and the standard grayscale image respectively. The histogram of the grayscale image to be matched is equalized to obtain a transformation function s k =T(r k ), where s k is the gray level after equalization, r k is the original grayscale, k represents the grayscale index of the histogram of the grayscale image to be matched before and after equalization, and T() represents the transformation function. At the same time, the histogram of the standard grayscale image is equalized to obtain a transformation function v q =G(z q ), where v q is the gray level after equalization, z q is the grayscale of the standard image, q represents the grayscale index of the histogram of the standard grayscale image before and after equalization, and G() represents the transformation function.

[0092] Ideally, all of the above are equalizations of the same treatment area, and the results should be equivalent:

[0093] s k =v q

[0094] Because: q =G(z q )

[0095] Through the inverse transformation, the following formula can be obtained:

[0096]

[0097] Among them, G -1 () represents the inverse transformation of the transformation function G().

[0098] By equalizing like this as an intermediate result, the original grayscale r can be obtained k and the normalized gray level z q In practical applications, it is not necessary to calculate the inverse transformation of G. Since the grayscale is an integer, such as the grayscale of an 8-bit image is 0-255, using v q =G(z q ) is relatively simple to calculate all values ​​when q=0,1,2,...,L-1 (L=255), then the following steps can be used to implement histogram matching:

[0099] Step 1: Equalize the grayscale histogram of the image to be matched. k Round it to an integer in [0, L-1]; at the same time, the histogram of the standard image is equalized, and v qThe value of is rounded to an integer in [0, L-1].

[0100] Step 2: For each s k value, we look for the corresponding v q value, so that v q Closest to k , and then we can get s k to z q When s is satisfied k z q When there are more than one value, the smallest z is selected. q The values ​​are mapped. k With z q The mapping relationship can be used to map the grayscale of the image to be matched after grayscale histogram equalization to the grayscale of the standard image to form a matched image.

[0101] The above is the algorithm of histogram matching. It can be seen that although histogram matching can make the visual effects of the images before and after treatment tend to be consistent, which is convenient for comparing the treatment effect, the introduction of grayscale conversion will inevitably cause some detail distortion. Therefore, this embodiment first adjusts the light source parameters to improve the histogram matching degree of the images before and after treatment, and then selects the images after treatment with the highest matching degree for histogram matching before and after treatment, which can reduce the adjustment range of the histogram, thereby reducing the detail distortion caused by directly using the images after treatment and the images before treatment for histogram matching.

[0102] Furthermore, during the treatment process, the histogram of the standard image in each treatment can also be recorded. and light source parameters R1, the histogram of the first image to be matched and light source parameter R2, and the adjustment value ⊿R of the light source parameter of the image to be matched with the highest matching degree relative to the first image to be matched, as historical data. , R1, , R2] as the state variable, ⊿R as the action variable, and the historical state variables and historical action variables in the same treatment as samples, and a sample library is constructed to learn the relationship between the state variable and the action variable. After learning, in the new treatment, a new standard image can be taken before the treatment, and only a new image to be matched needs to be taken after the treatment. The histograms of these two images and the light source parameters when they are taken together constitute the new state variable; the new state variable is substituted into the learned relationship to obtain the optimal action variable; according to the optimal action variable, the light source parameters when shooting the new image to be matched are adjusted, and the adjusted light source parameters are the optimal light source parameters. The image to be matched taken under this parameter is the image to be matched with the highest matching degree with the new standard image. The image is histogram matched with the new standard image, and the matched image can maintain the same visual effect as the new standard image, which is convenient for comparing the effects before and after treatment. It should be noted that the time span of "one treatment" here can be long or short, and can refer to a continuous treatment operation of a physician, or a treatment course, and this embodiment does not make specific restrictions.

[0103] Optionally, since the accumulation of treatment data requires a long time, the problem of insufficient samples is prone to occur during the learning process. In order to overcome this problem, this embodiment learns the relationship between state variables and action variables through the clustering characteristics of the sample set, wherein the role of clustering has two aspects: on the one hand, it simplifies the relationship between state variables and action variables, that is, the relationship between each state variable and each action variable is simplified to the relationship between each type of state variable and each type of action variable, so as to reduce the difficulty of learning; on the other hand, after clustering, each type of variable can include a certain amount of historical samples, reducing the adverse effects of missing samples of a certain type or unbalanced samples. In a specific embodiment, taking the adjusted light source parameters of brightness and color temperature as an example (other light source parameter combinations are similar and will not be repeated one by one), the above-mentioned use of the clustering characteristics of the sample set to learn the relationship between state variables and action variables can include the following steps:

[0104] Step 1: Cluster various value combinations of action variables to obtain multiple clusters. Compared with state variables, action variables have fewer parameters (including two parameters, brightness adjustment value and color temperature adjustment value), and the value range of action space is limited. Therefore, this embodiment clusters action variables. Optionally, the value ranges of brightness and color temperature can be divided into different intervals. For example, the most commonly used brightness and color temperature in historical data can be selected, and these commonly used values ​​can be used as the center to divide the intervals according to the rule of dense in the middle and sparse at both ends; the midpoint of each brightness interval is combined with the midpoint of each color temperature interval, and each combination is a value combination of light source parameters, thereby obtaining all value combinations of light source parameters; from all value combinations of light source parameters, any two combinations can be selected to make a difference (brightness and brightness are subtracted, color temperature and color temperature are subtracted), and a value combination of action variables can be obtained; all pairwise combinations of light source parameters are exhausted to obtain all value combinations of action variables. Clustering all value combinations of action variables can obtain multiple clusters. The specific clustering method can be selected Clustering, Clustering, etc. After clustering, the clusters satisfy the characteristics of minimizing the intra-class combination differences and maximizing the inter-class combination differences.

[0105] Step 2: Build a fully connected layer, which is used to extract the implicit features of the state variables. Due to the insufficient number of samples or the interference between the elements in the state variables, the historical state variables corresponding to the historical action variables belonging to the same cluster may not satisfy the same cluster relationship, that is, in the action variables A1, A2…A w When they are similar and can be grouped into one category, the historical state variables S1, S2…S corresponding to these action variables are w They do not show similar characteristics and cannot be directly clustered into one category. Therefore, this embodiment constructs a fully connected layer, aiming to mine the implicit features in the historical state variables by means of feature expansion or dimensionality reduction, and learn the clustering rules between state variables through the implicit features.

[0106] Step 3: Input the historical state variables corresponding to the action variables included in each cluster in the sample set into the fully connected layer respectively to obtain the implicit features of each historical state variable; by minimizing the difference of implicit features corresponding to the same cluster and maximizing the difference of implicit features corresponding to different clusters, the fully connected layer is trained so that the implicit features obtained by the trained fully connected layer can show the clustering relationship corresponding to the action variables, that is, if the action variables A1, A2…A w are clustered into one category, then the state variables S1, S2…S corresponding to these action variables w The implicit features of can also be clustered into one category. Specifically, the input data of the fully connected layer is the state variable [ , R1, , R2], we can and The values ​​in the N grayscale intervals are arranged in a sequence in increasing order of grayscale. (u = 1 or 2, indicating the histogram index), the two parameters in R1 and R2 are also arranged into a two-dimensional sequence [⊿L u ,⊿T u ], where ⊿L u and ⊿T u Respectively represent R u The brightness adjustment value and color temperature adjustment value in the [ , R1, , R2] are arranged in the order of state variables [ ,⊿L1,⊿T1, ,⊿L2,⊿T2]. For any cluster, multiple historical action variables that fall within the value combination range included in the cluster can be retrieved from the sample set; then the historical state variables corresponding to each retrieved historical action are extracted from the sample set. Thus, a set of historical state variables of each cluster is obtained, and a part of them is selected as a training set and a part as a test set. The historical state variables in the training set are input into the fully connected layer in batches to train the parameters within the layer, and the parameters are updated through the following loss function:

[0107] +

[0108] Among them, Loss represents the loss function value, n and m are the indexes of the clusters, and m is not equal to n; i and j are the indexes of the historical action variables in the same cluster. It represents the implicit features of the historical state variable corresponding to the historical action variable with index i in the cluster with index n. and The meaning is similar to; express and The vector similarity of The meaning of is similar. By minimizing the loss, the difference of implicit features corresponding to the same cluster can be minimized and the difference of implicit features corresponding to different clusters can be maximized, so as to reflect the clustering relationship between state variables through implicit features.

[0109] Step 4: After the training is completed, the historical state variables corresponding to each cluster cluster are respectively input into the trained fully connected layer to obtain the final implicit features of each historical state variable. At this time, the final implicit features corresponding to the same cluster cluster can constitute a new cluster cluster, and the cluster centers of these new cluster clusters are also fixed. For the convenience of distinction and description, in this embodiment, the cluster cluster of action variables in step 1 is called the first cluster cluster, and the cluster cluster of implicit features in this step is called the second cluster cluster. The second cluster cluster reflects the clustering characteristics of the state variables through implicit features. Each first cluster cluster corresponds to a second cluster cluster. This correspondence and the trained fully connected layer jointly reflect the relationship between the state variable and the action variable. Optionally, after the training of the fully connected layer in step 3 is completed, the test set can be divided into multiple batches, and each historical state variable in each batch is input into the trained fully connected layer to obtain each implicit feature; these implicit features correspond to multiple second cluster clusters, and the cluster center of each second cluster cluster jointly constitutes a group of cluster centers of the batch. Compare a group of cluster centers of each batch with the cluster centers of each implicit feature obtained by inputting each historical state variable in the training set into the trained fully connected layer. If the difference between each group of cluster centers is within the set range, it indicates that the fully connected layer extracts clustering characteristics more accurately, and the fully connected layer is usable. Otherwise, retraining is required.

[0110] After obtaining the cluster center of the second cluster and the trained fully connected layer in the above manner, these cluster centers and fully connected layers are fixed. In the new treatment, a new standard image can be taken before the treatment, and only a new image to be matched needs to be taken after the treatment. The new state variable can be obtained from the new standard image and the new image to be matched, and the new state variable is input into the trained fully connected layer to obtain a new latent feature. Then, the distance between the new latent feature and the cluster center of each second cluster is calculated, and the second cluster with the closest distance is used as the cluster cluster of the new latent feature. The first cluster corresponding to the cluster is recorded as A, and the value combination of the action variable closest to the cluster center of A is extracted as the optimal action variable.

[0111] In summary, this embodiment records the histogram and light source parameters of the standard image during each treatment, the histogram and light source parameters of the first image to be matched, and the adjustment value of the light source parameter with the highest matching degree, and uses the first four items as state variables and the last item as action variables to learn the relationship between state variables and action variables. After learning, in a new treatment, the amount by which the light source needs to be adjusted can be directly obtained based on the first four items of data, and the best histogram matching image can be obtained. In particular, in order to overcome the problem of insufficient medical data samples, this embodiment simplifies the relationship between state variables and action variables by clustering, and evenly distributes limited samples among various variables to improve the accuracy of learning, thereby improving the accuracy of light source parameter control and image histogram matching.

[0112] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An image artificial intelligence processing method, characterized in that: Based on an oral multi-specialty endoscope, wherein the oral multi-specialty endoscope comprises: an insertion portion (1), an operation portion (2), an operation button (3), an image acquisition element (11), a signal line (4) and an optical fiber (12); The front end of the operating part (2) is connected to the insertion part (1), the front end of the insertion part (1) is provided with an image acquisition element (11), the image acquisition element (11) is connected to the signal line (4), and the signal line (4) is connected to the host; A light source is disposed inside the operating portion (2) or in the main unit. When the light source is located in the main unit, light is transmitted to the insertion portion (1) through the optical fiber (12); The mirror tube (13) of the insertion part (1) is in the shape of a hook, a bend, a straight rod, or other shapes that can be observed; The method comprises: Calculating the histogram matching degree of a plurality of images to be matched and a standard image respectively, wherein the standard image is an image of the treatment area before treatment, and each image to be matched is an image of the treatment area after treatment under different light source parameters of the oral multi-specialty endoscope; The image to be matched with the highest matching degree is selected, and histogram matching is performed with the standard image. The matched image is used to demonstrate the treatment effect together with the standard image.

2. The image artificial intelligence processing method according to claim 1, characterized in that: Also includes: The historical state variables and historical action variables in the historical data are used as samples, and the sample set is used to learn the relationship between the state variables and the action variables; wherein the state variables are the histogram and light source parameters of the standard image before treatment, and the histogram and light source parameters of the first image to be matched after treatment, and the action variables are the light source parameter adjustment values ​​of the image to be matched with the highest final matching degree after treatment relative to the first image to be matched; A new standard image and a new image to be matched are used together to form a new state variable, and the learned relationship is substituted into the new state variable to obtain the optimal action variable. Another new image to be matched is obtained by adjusting the light source parameters according to the optimal action variable, and the new standard image is histogram matched. The matched image is used to demonstrate the treatment effect together with the new standard image.

3. The image artificial intelligence processing method according to claim 2, characterized in that: The method of using the sample set to learn the relationship between the state variable and the action variable includes: Clustering various value combinations of the action variables to obtain multiple first clusters; Inputting the historical state variables corresponding to the action variables included in each first cluster in the sample set into the fully connected layer respectively, obtaining the implicit features of each historical state variable, and training the fully connected layer by minimizing the difference of implicit features corresponding to the same first cluster and maximizing the difference of implicit features corresponding to different first clusters; Input the historical state variables corresponding to each first cluster into the trained fully connected layer respectively to obtain the final implicit features of each historical action variable, and use the final implicit features corresponding to the same first cluster as the second cluster; The corresponding relationship between each second cluster and each first cluster, and the trained fully connected layer are used together to characterize the relationship between the state variable and the action variable.

4. The image artificial intelligence processing method according to claim 1, characterized in that: The oral multi-specialty endoscope is made of hard material, and a jacket (5) is added to the outside of the insertion part (1), and the jacket (5) is made of flexible material.

5. The image artificial intelligence processing method according to claim 1, characterized in that: The operating part (2) is provided with an operating button (3), and the operating button (3) is used to adjust light source parameters and control shooting operations.

6. The image artificial intelligence processing method according to claim 1, characterized in that: The operating portion (2) is provided with a multi-purpose hollow channel (6), and the multi-purpose hollow channel (6) penetrates the insertion portion (1); The multi-purpose hollow channel (6) is used to inject flushing liquid and / or medicine into oral tissue, or to use a laser optical fiber to treat the affected area through the channel.

7. The image artificial intelligence processing method according to claim 1, characterized in that: An image processing unit is added to the operating part (2) according to clinical needs, and the image processing unit is used to process images.

8. The image artificial intelligence processing method according to claim 1, characterized in that: Also includes: Obtaining an image of the patient's oral cavity; The oral images are processed using AI algorithm models and medical image algorithms to automatically identify oral diseases, including dental pulp, root canal, mucosal, periodontal, salivary gland, and implant cavity diseases.

9. The image artificial intelligence processing method according to claim 8, characterized in that: Before the oral image is processed by using the AI ​​algorithm model and the medical image algorithm to automatically identify oral diseases, the method further includes: Collect oral image data and construct an oral image dataset; Use transfer learning methods to train the teacher model; According to the knowledge distillation method, the trained teacher model is used to guide the student model, and the trained student model is used as the final AI algorithm model. The model outputs are oral tissue type, lesion area block diagram and disease type.

Images