An intelligent analysis method for oral images

Abstract

The present application discloses an intelligent analysis method for oral images, including: obtaining a first oral image acquired by an oral image acquisition device under initial state parameters, and dividing it into a plurality of unit images, generating a coding group for each unit image according to a preset coding rule based on the relative position relationship between each unit image and a first reference line in the first oral image; obtaining a target oral image and its target reference line acquired by the oral image acquisition device under various target state parameters; generating all unit images and their coding groups for each target oral image based on each target oral image and its target reference line; fusing all unit images with the same coding group to obtain a panoramic unit image corresponding to each coding group, and obtaining a user's oral comprehensive index according to an oral morphology analysis algorithm and an oral appearance analysis algorithm. Thus, it is possible to differentiate different tooth information, combine oral morphology and oral appearance, and provide diversified oral condition analysis results.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an intelligent analysis method for oral images. Background Art

[0002] In recent years, with the continuous development of digitalization in the oral field, oral image acquisition has become easier and easier. Analyzing oral problems by collecting oral images is more intuitive and more effective than using other sensors. At present, the acquisition of oral images, such as using an electronic endoscope to reach into the mouth through a macro camera to collect images of the oral cavity at close range. However, further analysis of the oral condition of the image, such as determining whether the teeth are neat or defective, detecting tooth color, gum color, etc., generally requires professionals to analyze the image. Existing intelligent models for detecting tooth wear generally require the use of professional three-dimensional scanning equipment to obtain images in the oral cavity. The required equipment is expensive and inconvenient for daily use.

[0003] The invention patent with application number 202010363987.9 discloses an intelligent analysis device for oral images, which uses an electronic endoscope to collect oral images, performs image segmentation on the oral images to obtain tooth images and gum images, and performs feature extraction on the teeth and gums respectively. The deep convolutional neural network model under the pre-trained tooth aesthetics detection targets (tooth color, tooth arrangement, whether there are tooth defects, whether there are missing teeth, gum color) is used to obtain the fusion features of the oral image, which are input into the SVM classifier corresponding to each detection target to obtain the grade value corresponding to each tooth aesthetics detection target. According to the grade and weight of each target, the weighted sum is taken to obtain the oral aesthetics index.

[0004] However, the captured tooth images and gum images are not combined with the shooting position information of the image acquisition device. The feature matrix of the entire tooth image and gum image is input into the pre-trained model to judge the detection target level. The average condition of all teeth is obtained, and the differentiated characteristics between different teeth are ignored. There is a lack of multi-angle position information of each tooth relative to the entire oral cavity, and the healthy and aesthetic condition of the oral cavity cannot be fully and accurately analyzed. In addition, if the entire tooth image is used for model training, a large number of training sets with different tooth aesthetic distribution conditions are required for training if more accurate judgment results are to be obtained. The entire tooth image needs to be labeled with a grade value in the training set. Since the aesthetic condition of the entire tooth image needs to be taken into consideration, the labeling process is time-consuming and laborious. Summary of the invention

[0005] The present application provides an intelligent analysis method for oral images, which can differentiate different tooth information, combine oral morphology and oral appearance, and provide diversified oral condition analysis results.

[0006] The present application provides an intelligent analysis method for oral images, comprising:

[0007] S101, obtaining a first oral cavity image captured by an oral cavity image acquisition device under initial state parameters, positioning the oral cavity image acquisition device at a target position in the oral cavity for capturing an oral cavity image of a user, and generating a first reference line of the first oral cavity image;

[0008] S102, using image segmentation technology to segment the first oral cavity image into a plurality of unit images, each unit image including a tooth and its associated gum image information, and generating a coding group for each unit image according to a relative position relationship between each unit image and a first reference line in the first oral cavity image and a preset coding rule;

[0009] S103, obtaining a target oral cavity image acquired by the oral cavity image acquisition device under each target state parameter, and determining a target baseline of the target oral cavity image based on each target oral cavity image and its corresponding target state parameter;

[0010] S104, based on each target oral cavity image and its target baseline, executing step S102 to generate all unit images of each target oral cavity image and their coding groups;

[0011] S105, based on the unit images generated by the first oral cavity image and each target oral cavity image and their coding groups, all unit images with the same coding group are fused using an image fusion technology to obtain a panoramic unit image corresponding to each coding group;

[0012] S106, based on all panoramic unit images, according to the preset oral morphology analysis algorithm and oral appearance analysis algorithm, the oral morphology index and oral appearance index of the user are obtained, and weighted summation is performed to generate an oral comprehensive index.

[0013] Preferably, the oral cavity collection device is configured as a rotatable shooting rod, the end of which is equipped with a high-definition camera, a miniature fill light, and a position sensor;

[0014] The position sensor is used to locate the target position and generate corresponding state parameters, which include the rotation angles of the camera in the horizontal and vertical directions;

[0015] The target position is set as: a preset distance in a direction perpendicular to the straight line where the center seam of the lower front teeth is located in the oral cavity;

[0016] The initial state parameters of the oral image acquisition device are: the camera focus point is facing the center gap of the lower incisors, and the rotation angle is 0;

[0017] The first reference line of the first oral cavity image is set as: a straight line in a vertical direction from the camera focus point to the first oral cavity image.

[0018] Preferably, the preset coding rule is specifically:

[0019] A1. Establish a coordinate system with the first reference line as the y-axis and the dividing line between the upper oral cavity and the lower oral cavity as the x-axis;

[0020] A2. Traverse all unit images along the positive and negative directions of the x-axis and the positive and negative directions of the y-axis respectively, assign coding values ​​of the unit images on the x-axis and y-axis in sequence according to increasing or decreasing serial numbers, and generate a coding group (x, y) of the unit images.

[0021] Preferably, in S103, based on each target oral image, determining a target baseline of the target oral image according to its corresponding target state parameter includes:

[0022] B1. Obtain a straight line in the vertical direction from the focus point of the camera to the target oral cavity image when shooting under the target state parameters, as the initial reference line;

[0023] B2. Obtain the rotation angle in the horizontal direction of the target state parameter, invert it to obtain the target angle, rotate the initial reference line by the target angle, and obtain the target reference line of the target oral cavity image.

[0024] Preferably, the oral morphology index of the user is obtained according to a preset oral morphology analysis algorithm, specifically including:

[0025] S201, arranging the positions of the panoramic unit images according to their coding groups, and splicing the two adjacent panoramic unit images by using image fusion registration technology to generate a panoramic oral image;

[0026] S202, based on the panoramic oral image, extract the first offset sequence and the second arrangement curve of the teeth, input them into the pre-trained tooth protrusion recognition model, and output the tooth protrusion posture grade value and the tooth protrusion posture label; the tooth protrusion posture label includes: normal, protrusion, and intrusion;

[0027] S203: Outputting the oral morphology index of the user according to the dental protrusion posture level value of the panoramic oral image.

[0028] Preferably, the panoramic unit images are all annotated with tooth types according to their coding groups, and the tooth types include incisors, canines, premolars, and molars;

[0029] The oral appearance analysis algorithm is used to pre-train the oral recognition model corresponding to each oral detection target, including a feature extraction model and a classification model. The output end of the feature extraction model is connected to the input end of the classification model, specifically including:

[0030] S301, attaching corresponding tooth type labels to each panoramic unit image, respectively inputting them into the oral recognition models corresponding to each oral detection target that has been pre-trained, and outputting the grade value of the panoramic unit image on each oral detection target;

[0031] S302, calculating the user's oral appearance index according to the following formula:

[0032] ;

[0033] Where P is the user's oral appearance index, n is the number of the user's panoramic unit images, and m is the total number of oral detection targets. is the preset weight coefficient value of the ith panoramic unit image on the jth oral detection target, which is used to comprehensively reflect the importance of the oral detection target and the tooth type of the coding group of the panoramic unit image in the oral cavity to the user's oral appearance. is the level value of the i-th panoramic unit image on the j-th oral detection target.

[0034] Preferably, in S202, based on the panoramic oral image, extracting the first offset matrix and the second arrangement curve of the teeth specifically includes:

[0035] S401, determining the center point of all panoramic unit images in the panoramic oral image;

[0036] S403, according to a preset direction, respectively obtain the rays where the center points of two adjacent panoramic unit images are connected, and sequentially obtain the angle values ​​formed by the two adjacent rays until all panoramic unit images are traversed to obtain the upper tooth offset sequence and the lower tooth offset sequence;

[0037] S404, combining the upper tooth offset sequence and the lower tooth offset sequence to obtain a first offset matrix;

[0038] S405, acquiring a gum image in the panoramic oral image, and generating a gum simulation curve according to the gum image;

[0039] S406, obtaining a dental arch image in the panoramic oral image, performing a texture analysis technique on the dental arch image, extracting texture information from the dental arch image, and generating a tooth arrangement curve using a curve connection technique;

[0040] S407, combining the gum simulation curve and the tooth arrangement curve to form a second arrangement curve.

[0041] Preferably, the oral morphology analysis algorithm further includes:

[0042] S501, based on the panoramic oral image, using image segmentation technology to segment the panoramic oral image into a plurality of associated tooth groups, where the associated tooth group includes two adjacent tooth images;

[0043] S502, based on each associated tooth group, using feature matching technology, identifying a target feature point group of each tooth image, where the target feature point group includes any of the tips, edge intersections, and inflection points of the tooth image;

[0044] S503, based on the target feature point group of each tooth image, obtain a line segment connecting any two target feature points therein, take the line segment with the largest span on the tooth image as the first major axis of the tooth image, and obtain the angle value between the first major axis and the straight line where the extension direction of the tooth image is located;

[0045] S504, determining whether the angle value is greater than the deviation threshold, if not, determining the first long axis as the target long axis of the tooth image, if yes, repeating step S503 to re-determine other long axes different from the first long axis, and determining the angle value until the angle value is less than the deviation threshold;

[0046] S505, determining the target long axis of each tooth image in the associated tooth group, and calculating the torsion angle value of the associated tooth group according to the following formula:

[0047] ;

[0048] in, and Represents the target long axis vector of each tooth image, is the torsion angle value of the associated tooth group;

[0049] S506, according to the torsion angle values ​​of all associated tooth groups, the comprehensive torsion angle value of the panoramic oral image is calculated according to the following formula:

[0050] ;

[0051] in, is the comprehensive torsion angle value, K is the total number of associated tooth groups, is the torsion angle value of the kth associated tooth group, is the similarity of the tooth types of different tooth images in the kth associated tooth group;

[0052] S507: According to the comprehensive torsion angle value of the panoramic oral image and in combination with a plurality of pre-set angle intervals, a tooth alignment grade value of the user is given.

[0053] Preferably, in S203, outputting the oral morphology index of the user according to the dental protrusion posture level value of the panoramic oral image specifically includes:

[0054] The tooth protrusion posture grade value and the tooth alignment grade value of the panoramic oral image are obtained, and weighted summation is performed to obtain the oral morphology index of the user.

[0055] Preferably, in S501, the method further includes:

[0056] S601, collecting a tooth gap image formed between two tooth images in an associated tooth group to obtain an associated tooth gap image;

[0057] S602, extracting a first feature data group associated with the interdental image, the first feature data group including a color histogram, a shape descriptor, and a texture descriptor associated with the interdental image;

[0058] S603, obtaining a second feature data set of a pre-prepared dental calculus sample image, the second feature data set including a color histogram, a shape descriptor, and a texture descriptor of the dental calculus sample image;

[0059] S604, traversing and comparing the first feature data group and the second feature data group, and determining the connected domain area in the associated interdental image whose similarity reaches the similarity standard as the suspected calculus area;

[0060] S605, inputting the suspected calculus area into the pre-trained calculus recognition model, outputting a probability distribution map of calculus, obtaining an area with a probability value greater than 90% and forming a connected domain in the probability distribution map, and determining it as a calculus image, otherwise, there is no calculus in the associated interdental image;

[0061] S606, using image registration and stitching technology, determine the tooth image that matches the tartar image according to the position and shape of the tartar image in the associated interdental image, and stitch the tartar image to the corresponding position of the matching tooth image to update the associated tooth group.

[0062] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0063] By introducing the concepts of position sensors and baselines, the image acquisition device is used to collect oral images from multiple angles at a fixed position in the oral cavity, and the panoramic unit images are fused to ensure a comprehensive analysis of each tooth and its associated gums; based on each panoramic unit image, the target level value of each oral cavity is output respectively, and based on the panoramic oral image, the overall dental protrusion posture level value is output, thereby realizing the oral morphology and oral appearance, accurately analyzing and evaluating the user's oral cavity, and synchronizing the panoramic oral image and its annotation information on the terminal, so that the user can have a more comprehensive understanding of the problems and conditions in his or her oral cavity;

[0064] By further elaborating on the oral morphology analysis algorithm and introducing the concepts of the first offset matrix and the second arrangement curve, a more refined analysis of the offset and arrangement of teeth was conducted, which can not only reflect the overall arrangement of teeth, but also specifically point out which areas of teeth have offset or misaligned problems; through the tooth protrusion recognition model, combined with the first offset matrix and the second arrangement curve as input features, the level and type of tooth protrusion posture can be more accurately identified;

[0065] For the determination of oral morphology index, the concept of associated tooth group is introduced to conduct a more refined analysis of the relationship between adjacent teeth. By calculating the torsion angle value of the associated tooth group, the relative position and torsion between teeth can be more accurately reflected, providing richer information for oral morphology analysis.

[0066] In the process of image processing of the associated tooth group, by introducing the steps of tartar recognition and processing, the problem of missing tooth images caused by tartar is effectively solved, the suspected tartar area is accurately identified, and the tartar recognition model is used for further judgment to ensure the accurate position and range of the tartar. The identified tartar image is spliced ​​to the corresponding position of the matching tooth image, thereby updating the associated tooth group, ensuring the integrity of each tooth image, and providing more reliable image support for subsequent target long axis calculation and oral morphology analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 Schematic diagram of the process of the intelligent analysis method of oral images according to an embodiment of the present invention. DETAILED DESCRIPTION

[0068] To facilitate the understanding of the present invention, the present application will be described more comprehensively below with reference to the relevant drawings; the drawings show preferred embodiments of the present invention, but the present invention can be implemented in many different forms and is not limited to the embodiments described herein; on the contrary, the purpose of providing these embodiments is to enable a more thorough and comprehensive understanding of the disclosed content of the present invention.

[0069] It should be noted that the terms “vertical”, “horizontal”, “up”, “down”, “left”, “right” and similar expressions used in this document are only for illustrative purposes and do not represent the only implementation method.

[0070] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field to which the present invention belongs; the terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention; the term "and / or" used herein includes any and all combinations of one or more related listed items.

[0071] Embodiment 1: Figure 1 It is a flowchart of an intelligent analysis method for oral images according to an embodiment of the present invention.

[0072] like Figure 1 As shown, an intelligent analysis method for oral images is applied to a terminal that establishes wireless communication with an oral image acquisition device, comprising the following steps:

[0073] S101, obtaining a first oral cavity image captured by an oral cavity image acquisition device under initial state parameters, positioning the oral cavity image acquisition device at a target position in the oral cavity for capturing an oral cavity image of a user, and generating a first reference line of the first oral cavity image.

[0074] Among them, the oral acquisition device is set as a rotatable shooting rod, and the end is equipped with a high-definition camera, a miniature fill light, and a position sensor. It has a wide-angle shooting function and can collect images of the upper and lower oral cavity; the position sensor is used to locate the target position and generate corresponding state parameters. The state parameters include the camera rotation angle (horizontal direction, vertical direction), wherein the rotation angle range in the horizontal direction and the rotation angle range in the vertical direction are set to [-45°, 45°], and the camera is rotated to the right in the horizontal direction relative to the initial state parameters. The angle is defined as [0, 45°], and the camera is rotated to the left in the horizontal direction relative to the initial state parameters. The angle is defined as [-45°, 0]; the upward rotation angle of the camera in the vertical direction relative to the initial state parameters is defined as [0, 45°], and the downward rotation angle of the camera in the vertical direction relative to the initial state parameters is defined as [-45°, 0].

[0075] Specifically, the target position is set to: a preset distance in a direction perpendicular to the straight line where the center seam of the lower incisors is located when the target position is extended into the oral cavity. The preset distance can be set according to actual conditions. The initial state parameters of the oral image acquisition device are: the camera focus point is facing the center seam of the lower incisors, and the rotation angle is 0. The first reference line of the first oral image is set to: the camera focus point is positioned to the straight line in the vertical direction of the first oral image.

[0076] S102, using image segmentation technology to segment the first oral image into a number of unit images, each unit image including a tooth and its associated gum image information, and generating a coding group for each unit image according to a relative position relationship between each unit image and a first baseline in the first oral image and a preset coding rule.

[0077] Among them, the associated gum image information is set to the gum image information corresponding to the left and right sides of the teeth. The image segmentation technology can use existing image segmentation processing methods (for example, pixel threshold segmentation, edge detection, deep learning segmentation model, etc.) to achieve the segmentation of all teeth and their associated gum images in the entire oral image. The present invention does not elaborate on this.

[0078] Specifically, the preset encoding rules are:

[0079] A1. Establish a coordinate system with the first reference line as the y-axis and the dividing line between the upper oral cavity and the lower oral cavity as the x-axis.

[0080] A2. Traverse all unit images along the positive and negative directions of the x-axis and the positive and negative directions of the y-axis respectively, assign coding values ​​of the unit images on the x-axis and y-axis in sequence according to increasing or decreasing serial numbers, and generate a coding group (x, y) of the unit images.

[0081] For example, the encoding value of the unit image in the positive direction of the y-axis on the y-axis is set to +1, the encoding value of the unit image in the negative direction of the y-axis on the y-axis is set to -1, the encoding value of the unit image in the positive direction of the x-axis on the x-axis is set to +n, and the encoding value of the unit image in the negative direction of the x-axis on the x-axis is set to -n, n is set to (1, 2, ..., n), the value of n depends on the sequence number of the unit image on one side of the y-axis, and the number is increased from the y-axis to one side to obtain the value of n.

[0082] S103, obtaining a target oral cavity image acquired by the oral cavity image acquisition device under each target state parameter, and determining a target baseline of the target oral cavity image based on each target oral cavity image and its corresponding target state parameter.

[0083] Among them, each target state parameter is preset, and the rotation angle of the camera is set according to the specific shooting requirements, so as to obtain multi-angle oral images; specifically, based on each target oral image, the target baseline of the target oral image is determined according to its corresponding target state parameter, including:

[0084] B1. Obtain a straight line in the vertical direction where the focus point of the camera is located to the target oral cavity image when shooting under the target state parameters, as the initial reference line.

[0085] B2. Obtain the rotation angle in the horizontal direction of the target state parameter, invert it to obtain the target angle, rotate the initial reference line by the target angle, and obtain the target reference line of the target oral cavity image.

[0086] S104, based on each target oral cavity image and its target baseline, executing step S102 to generate all unit images of each target oral cavity image and their coding groups.

[0087] It should be noted that the first oral cavity image and the first reference line in step S102 can be replaced by the target oral cavity image and the target reference line to generate all unit images and encoding groups of each target oral cavity image, and the present invention will not elaborate on this principle.

[0088] S105, based on the unit images and their coding groups generated by the first oral image and each target oral image, all unit images with the same coding group are fused using image fusion technology to obtain a panoramic unit image corresponding to each coding group. The number of coding groups corresponds to the total number of teeth in the user's mouth.

[0089] Among them, each panoramic unit image is labeled with the tooth type according to its coding group. Since the arrangement position of teeth in the human mouth is fixed, different position ranges correspond to different tooth types. The tooth type of each panoramic unit image can be quickly located according to the human oral model (defined as distinguishing different types of tooth shapes). For example, tooth types include incisors, canines, premolars, and molars.

[0090] It should be noted that the image fusion technology can use existing image processing methods to combine the same tooth and its associated gum image information from multiple angles to obtain a complete tooth image information.

[0091] In the field of stomatology, this image fusion technology is widely used. Specifically, the unit images generated from different shooting angles contain image information of teeth and their associated gums from different perspectives, but each unit image can only provide local information of the teeth. In order to obtain a complete view of the teeth, these unit images need to be fused. During the fusion process, image fusion technology uses advanced algorithms to analyze and process all unit images, identify common features in each unit image, such as the edge and contour of the teeth, and match and dock these features, ensuring that these unit images taken from different angles can be combined to form a complete image of the teeth and their surrounding gums, that is, a panoramic unit image, which provides a complete view of the teeth and their surrounding gums. It not only contains images of the teeth from different angles, but also retains the details and clarity of the original unit images. Through the panoramic unit images, the morphology, position and relationship of the teeth with the surrounding gums can be understood more comprehensively and accurately, thereby providing more reliable information for diagnosis and treatment.

[0092] S106, based on all panoramic unit images, according to the preset oral morphology analysis algorithm and oral appearance analysis algorithm, the oral morphology index and oral appearance index of the user are obtained, and weighted summation is performed to generate an oral comprehensive index.

[0093] In some embodiments, the oral morphology index of the user is obtained according to a preset oral morphology analysis algorithm, specifically including:

[0094] S201, arranging the positions of the panoramic unit images according to their coding groups, and splicing the two adjacent panoramic unit images by using image fusion registration technology to generate a panoramic oral image.

[0095] Specifically, all panoramic unit images are standardized, including size adjustment, brightness and contrast adjustment, etc., and the processed panoramic unit images are arranged in positions using image fusion and registration technology (such as feature matching and image fusion algorithms) to form a complete panoramic oral image. The specific position arrangement can be based on the contents of steps A1 and A2.

[0096] S202, based on the panoramic oral image, extract the first offset sequence and the second arrangement curve of the teeth to represent the tooth process posture, input the first offset sequence and the second arrangement curve into a pre-trained tooth process recognition model, and output the tooth process posture grade value and the tooth process posture label.

[0097] S203: Outputting the oral morphology index of the user according to the dental protrusion posture level value of the panoramic oral image.

[0098] In some embodiments, the pre-set oral appearance analysis algorithm is used for the oral recognition model corresponding to each oral detection target that has been pre-trained, including a feature extraction model and a classification model. The output end of the feature extraction model is connected to the input end of the classification model, specifically including:

[0099] S301, attaching corresponding tooth type labels to each panoramic unit image, respectively inputting them into the oral recognition models corresponding to each oral detection target that has been pre-trained, and outputting the grade value of the panoramic unit image on each oral detection target.

[0100] Specifically, the process of pre-training the oral recognition model corresponding to each oral detection target includes:

[0101] C1. Collect sample data sets in advance for oral recognition model training.

[0102] When establishing an oral recognition model, the oral detection targets of interest include tooth color, tooth defects, and gum color. Therefore, for each oral detection target, a large number of oral images collected by oral image acquisition equipment are collected in advance, and the level value of each oral detection target is annotated for each image data to obtain an annotated sample data set; wherein, the annotation of the sample data set is based on the expert annotating the level value of each oral detection target on the collected oral images, and the annotated oral image data is used as the sample data set, wherein the range of the level value annotation is [0,5], and the larger the level value, the more serious the problem of the oral detection target;

[0103] When setting the grade value training rules, when training the oral cavity recognition model of tooth defects, different grade value division rules of tooth defects are set according to different tooth type labels. The larger the tooth defect, the higher the grade value. The farther the tooth is located in the oral cavity (determined according to the tooth type label), the higher the grade value. Thus, a large number of sample data sets used for training tooth defects are annotated with tooth type labels, and the oral cavity recognition model of tooth defects is trained according to different tooth type labels.

[0104] The training methods of classification models for other oral detection targets are similar, and the present invention will not elaborate on them.

[0105] C2. Based on each panoramic unit image, it is input into the feature extraction model of each oral detection target to extract oral features and generate a fusion feature matrix corresponding to the panoramic unit image.

[0106] Specifically, the feature extraction model is used to extract oral features, specifically:

[0107] When extracting oral features, the main focus is on features such as the color, texture, contour, and relative position of the gums and teeth, and different oral detection target problems have different degrees of focus on tooth and gum features. Therefore, the embodiment of the present invention extracts features of teeth and gums separately when extracting oral features. Before extracting tooth and gum features, an image segmentation algorithm is first used to segment each collected oral image into a tooth image and a gum image, and the image is scaled to obtain an image of 200*200 pixels in size, and then the segmented tooth image and the corresponding oral image annotation results constitute a tooth image training data set, and the segmented gum image and the corresponding oral image annotation results constitute a gum image training data set. Among them, the image segmentation algorithm can adopt a binarization processing method, which is not elaborated in the present invention.

[0108] Then, for each oral detection target, a preset deep convolutional neural network is used to extract the tooth feature matrix for the tooth image and the gum feature matrix for the gum image; taking the extraction of the tooth feature matrix as an example, the pictures in the tooth image training data set are used as input, and the tooth color annotation results are used as output vectors to train a deep convolutional neural network that can recognize the tooth color grade value, and obtain the feature extraction model corresponding to the tooth color.

[0109] Finally, the feature matrices of gums and teeth on the same oral image are fused to obtain a fused feature matrix.

[0110] Therefore, after the oral image is input into the feature extraction model, a fused feature matrix is ​​obtained, and after each panoramic unit image acquired by the oral image acquisition device is processed using the above steps, a fused feature matrix of each panoramic unit image can be obtained.

[0111] It should be noted that the fusion feature matrix of each image in the sample training set and its level value annotation results are combined into a feature data set for subsequent training, thereby obtaining a trained oral recognition model. For the construction and training methods of the model, reference can be made to the specific description in the relevant prior art, and the present invention will not elaborate on this.

[0112] C3. Input the fused feature matrix into the classification model corresponding to each oral detection target, and output the grade value of the panoramic unit image on each oral detection target.

[0113] S302, calculating the user's oral appearance index according to the following formula:

[0114] ;

[0115] Where P is the user's oral appearance index, n is the number of the user's panoramic unit images (i.e. the total number of teeth in the user's mouth), and m is the total number of oral detection targets. is the preset weight coefficient value of the ith panoramic unit image on the jth oral detection target, which is used to comprehensively reflect the importance of the oral detection target and the coding group of the panoramic unit image in the oral cavity (to distinguish different tooth types) to the user's oral appearance. is the level value of the i-th panoramic unit image on the j-th oral detection target.

[0116] The technical solutions in the above embodiments of the present application have at least the following technical effects or advantages:

[0117] By introducing the concepts of position sensors and baselines, the image acquisition device is used to collect oral images from multiple angles at a fixed position in the oral cavity, and the panoramic unit images are fused to ensure a comprehensive analysis of each tooth and its associated gums. Based on each panoramic unit image, the target grade value for each oral cavity detection is output respectively, and based on the panoramic oral image, the overall dental protrusion posture grade value is output, thereby realizing the oral morphology and oral appearance, accurately analyzing and evaluating the user's oral cavity, and synchronizing the panoramic oral image and its annotation information on the terminal, so that the user can have a more comprehensive understanding of the problems and conditions in his or her own oral cavity.

[0118] By segmenting unit images and fusing panoramic unit images, the model can be trained more comprehensively to obtain more accurate evaluation results for single teeth. Finally, a weighted evaluation is performed based on the type of tooth to obtain a health analysis result for the entire oral cavity.

[0119] Embodiment 2: In embodiment 1, although a comprehensive analysis of oral morphology and appearance has been achieved, the specific analysis of tooth arrangement and deviation is not in-depth enough, especially for the identification of the tooth protrusion posture, there is a lack of refined analysis and accurate identification methods. The tooth protrusion posture is an important indicator in oral health analysis. It reflects the arrangement and deviation of teeth and is of great significance for diagnosing tooth deformities and formulating correction plans. However, in traditional oral analysis methods, the analysis of the tooth protrusion posture often relies on the doctor's experience and subjective judgment, and lacks objective and accurate analysis tools.

[0120] Therefore, the embodiments of the present application are optimized to a certain extent based on the above embodiments.

[0121] In some embodiments, in step S202, extracting a first offset matrix and a second arrangement curve of teeth based on the panoramic oral image specifically includes:

[0122] S401, determining the center point of all panoramic unit images in the panoramic oral image.

[0123] The center point of each panoramic unit image is determined by: identifying the dental arch image in the panoramic unit image, obtaining the maximum circumscribed circle of the dental arch image, and determining the center of the maximum circumscribed circle as the center point of the panoramic unit image.

[0124] S403, according to the preset direction, respectively obtain the rays where the center points of two adjacent panoramic unit images are located, and obtain the angle values ​​formed by the two adjacent rays in turn, until all panoramic unit images are traversed, and the upper teeth offset sequence and the lower teeth offset sequence are obtained. The preset direction is set to be from the left side to the right side of the panoramic oral image.

[0125] Among them, the angle values ​​generated by traversing the panoramic unit image of the upper teeth constitute the upper tooth offset sequence, and the angle values ​​generated by traversing the panoramic unit image of the lower teeth constitute the lower tooth offset sequence.

[0126] For example, there are three adjacent panoramic unit images of the upper teeth, which are A, B, and C according to the preset direction. The rays connecting the center points of two adjacent panoramic unit images are obtained respectively. The rays are AB and BC respectively. The angle values ​​formed by the two adjacent rays are obtained in turn. Then this angle value corresponds to the angle formed by AB and BC. This angle value is matched as the offset angle of B.

[0127] S404: Combine the upper teeth offset sequence and the lower teeth offset sequence to obtain a first offset matrix.

[0128] Specifically, the first offset matrix has 2 rows and k columns, where k is the number of angle values ​​in the upper tooth offset sequence or the lower tooth offset sequence.

[0129] S405, acquiring a gum image in the panoramic oral image, and generating a gum simulation curve according to the gum image.

[0130] Specifically, image segmentation techniques (such as threshold segmentation, edge detection, etc.) are used to separate the gum area from the background, and the segmented gum image is subjected to morphological processing, such as filling holes, smoothing edges, etc., to obtain a more accurate gum contour. According to the contour of the gum image, curve fitting techniques (such as polynomial fitting, spline curves, etc.) are used to generate a gum simulation curve to ensure that the fitted curve can accurately reflect the shape and trend of the gums.

[0131] S406, obtaining a dental arch image (the occlusal surface of the teeth) in the panoramic oral image, performing texture analysis on the dental arch image, extracting texture information of the dental arch image, and generating a tooth arrangement curve using a curve connection technique.

[0132] Among them, texture analysis can use gray level co-occurrence matrix, local binary pattern and other methods to extract texture information, and curve connection technology can use dynamic programming, least square method and other methods to generate curves, which will not be elaborated in the present invention.

[0133] S407, combining the gum simulation curve and the tooth arrangement curve to form a second arrangement curve.

[0134] In some embodiments, in step S202, the pre-trained odontoid recognition model is trained by a method including:

[0135] Preparation: Collect a large number of panoramic oral images for training, and annotate the dental protrusion posture grade value of each panoramic oral image according to expert experience (the same as the grade value of the oral detection target, 0 to 5, the higher the grade value, the more serious the dental protrusion posture), and use the above steps S401 to S407 to obtain the first offset matrix and the second arrangement curve of each panoramic oral image respectively;

[0136] The odontoid process recognition model includes two input ports, which are respectively connected to a first feature extraction model and a second feature extraction model, and are respectively used to input a first offset matrix and a second arrangement curve of a panoramic oral image, and respectively output first feature information and second feature information. The output ports of the first feature extraction model and the second feature extraction model are commonly connected to an odontoid process detection classification model, and are used to output an odontoid process posture grade value and an odontoid process posture label corresponding to the panoramic oral image.

[0137] All panoramic oral images used for training are used to train the odontoid process recognition model to obtain a trained odontoid process recognition model.

[0138] It should be noted that the specific structure of the model used above can use the existing convolutional neural network structure, which will not be elaborated in the present invention.

[0139] Among them, the tooth protrusion posture labels include: normal, protrusion, and intrusion.

[0140] The technical solutions in the above embodiments of the present application have at least the following technical effects or advantages:

[0141] By further elaborating on the oral morphology analysis algorithm and introducing the concepts of the first offset matrix and the second arrangement curve, a more refined analysis of the offset and arrangement of teeth was conducted, which can not only reflect the overall arrangement of teeth, but also specifically point out which areas of teeth have offset or misalignment problems.

[0142] Through the odontoid recognition model, combined with the first offset matrix and the second arrangement curve as input features, the level and type of odontoid posture can be more accurately identified, which provides users with more specific and accurate oral morphology analysis results.

[0143] Example 3: In traditional oral morphology analysis, the analysis is limited to the dental protrusion posture and is often not accurate and comprehensive enough. At the same time, there is a lack of a unified indicator for the comprehensive evaluation of oral morphology. The alignment of teeth is also crucial in oral morphology analysis. Relying solely on the tooth arrangement curve is too limited. The relative position between single teeth is extremely critical.

[0144] Therefore, the embodiments of the present application are optimized to a certain extent based on the above embodiments.

[0145] In some embodiments, the pre-set oral morphology analysis algorithm further includes:

[0146] S501, based on the panoramic oral image, using image segmentation technology to segment the panoramic oral image into a number of associated tooth groups, where the associated tooth group includes two adjacent tooth images.

[0147] S502, based on each associated tooth group, using feature matching technology (eg, FLANN matcher), identifying a target feature point group of each tooth image, wherein the target feature point group includes any of the tips, edge intersections, and inflection points of the tooth image.

[0148] S503, based on the target feature point group of each tooth image, obtain the line segment connecting any two target feature points therein, take the line segment with the largest span on the tooth image as the first major axis of the tooth image, and obtain the angle value between the first major axis and the straight line where the extension direction of the tooth image is located.

[0149] S504, determine whether the angle value is greater than the deviation threshold (set according to actual conditions, for example, set to 5 degrees). If not, determine the first long axis as the target long axis of the tooth image. If so, repeat step S503 to re-determine other long axes different from the first long axis and determine the angle value until the angle value is less than the deviation threshold.

[0150] S505, determining the target long axis of each tooth image in the associated tooth group, and calculating the torsion angle value of the associated tooth group according to the following formula:

[0151] ;

[0152] in, and Represents the target long axis vector of each tooth image, is the torsion angle value of the associated tooth group;

[0153] S506, according to the torsion angle values ​​of all associated tooth groups, the comprehensive torsion angle value of the panoramic oral image is calculated according to the following formula:

[0154] ;

[0155] in, is the comprehensive torsion angle value, K is the total number of associated tooth groups, is the torsion angle value of the kth associated tooth group, is the similarity of the tooth types of different tooth images in the kth associated tooth group; for example, if the tooth types of the tooth images in a certain associated tooth group are very similar (such as two adjacent incisors), the weight value can be higher, because such associated tooth groups have a greater impact on the judgment of the comprehensive torsion angle value; on the contrary, if the tooth types of the tooth images in a certain associated tooth group are very different (such as an incisor and a molar), the weight value can be lower. The specific similarity value can be set according to the actual situation.

[0156] S507: According to the comprehensive torsion angle value of the panoramic oral image and in combination with a plurality of pre-set angle intervals, a tooth alignment grade value of the user is given.

[0157] Specifically, the range of grade values ​​is [0,5]. The larger the grade value, the more serious the problem of tooth alignment. 0 to 90° is evenly divided into 5 angle intervals, with one grade value corresponding to each angle from small to large.

[0158] In some embodiments, in step S203, outputting the oral morphology index of the user according to the dental protrusion posture level value of the panoramic oral image specifically includes:

[0159] The tooth protrusion posture grade value and the tooth alignment grade value of the panoramic oral image are obtained, and weighted summation is performed to obtain the oral morphology index of the user.

[0160] Among them, according to the influence of tooth protrusion posture and tooth neatness on oral morphology, weight coefficient values ​​are respectively assigned to the tooth protrusion posture grade value and the tooth neatness grade value, which are multiplied with the corresponding weight coefficient values ​​and then summed to obtain the oral morphology index.

[0161] The technical solutions in the above embodiments of the present application have at least the following technical effects or advantages:

[0162] For the determination of the oral morphology index, the concept of associated tooth groups is introduced to perform a more detailed analysis of the relationship between adjacent teeth. By calculating the torsion angle values ​​of associated tooth groups, the relative position and torsion between teeth can be more accurately reflected, providing richer information for oral morphology analysis. The oral morphology index not only takes into account the tooth protrusion posture, but also combines the tooth neatness for a comprehensive assessment. By calculating the comprehensive torsion angle value of the panoramic oral image and giving the tooth neatness grade value, a more comprehensive oral morphology analysis result is provided to users.

[0163] Embodiment 4: In embodiment 3, the extraction of target feature points of adjacent tooth images in an associated tooth group relies on the edge key points of each tooth image. However, in the process of tooth segmentation, since tartar may be partially confused with the gaps formed between adjacent teeth, part of the tartar belonging to the tooth may be mistakenly judged as being separated from the gaps, resulting in missing images of the teeth, causing errors in the subsequent selection of target feature points based on the entire tooth image.

[0164] Therefore, the embodiments of the present application are optimized to a certain extent based on the above embodiments.

[0165] In some embodiments, in step S501, the method further includes:

[0166] S601, collecting a tooth gap image formed between two tooth images in an associated tooth group to obtain an associated tooth gap image.

[0167] S602, extracting a first feature data group associated with the interdental image, the first feature data group including a color histogram, a shape descriptor and a texture descriptor associated with the interdental image.

[0168] Among them, the shape descriptor represents the edge length and curvature of the associated tooth gap image; the texture descriptor represents the gray-level co-occurrence matrix.

[0169] S603, obtaining a second feature data set of a pre-prepared dental calculus sample image, wherein the second feature data set includes a color histogram, a shape descriptor, and a texture descriptor of the dental calculus sample image.

[0170] S604: traverse and compare the first feature data group and the second feature data group, and determine the connected domain area in the associated interdental image whose similarity reaches the similarity standard as the suspected calculus area.

[0171] S605, input the suspected tartar area into the pre-trained tartar recognition model, output the probability distribution map of the tartar, in the probability distribution map, obtain the area with a probability value greater than 90% and forming a connected domain, and determine it as a tartar image, otherwise, there is no tartar in the associated interdental image.

[0172] In the process of training the tartar recognition model, a large number of interdental images and corresponding tartar annotation data are used as training sets to train the CNN model to obtain the tartar recognition model. Through the judgment of the tartar recognition model, the exact location and range of the tartar can be further determined, and the probability distribution diagram can show the probability of each location in the tartar area being tartar, so as to locate the tartar more accurately.

[0173] S606, using image registration and stitching technology, determine the tooth image that matches the tartar image according to the position and shape of the tartar image in the associated interdental image, and stitch the tartar image to the corresponding position of the matching tooth image, thereby updating the associated tooth group.

[0174] Thus, the integrity of each tooth image is ensured, including the possibly missing portion occupied by dental plaque, providing more reliable image support for the subsequent calculation of the target long axis of the tooth image.

[0175] The technical solutions in the above embodiments of the present application have at least the following technical effects or advantages:

[0176] In the process of image processing of the associated tooth group, the problem of missing tooth images that may be caused by dental calculus is effectively solved by introducing the steps of dental calculus recognition and processing. By extracting the feature data group of the associated interdental image and comparing it with the feature data of the dental calculus sample image, the suspected dental calculus area can be accurately identified, and then the dental calculus recognition model is used for further judgment to ensure that the accurate position and range of the dental calculus are determined.

[0177] Using image registration and stitching technology, the identified tartar image is stitched to the corresponding position of the matching tooth image, thereby updating the associated tooth group and ensuring the integrity of each tooth image, providing more reliable image support for subsequent target long axis calculation and oral morphology analysis; by solving the problem of missing tooth images caused by tartar, the accuracy of oral morphology analysis is further improved. The complete tooth image can more accurately reflect the relative position and torsion between teeth, providing richer information for oral morphology analysis.

[0178] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An intelligent analysis method for oral images, applied to a terminal that establishes wireless communication with an oral image acquisition device, characterized in that: include: S101, obtaining a first oral cavity image captured by an oral cavity image acquisition device under initial state parameters, positioning the oral cavity image acquisition device at a target position in the oral cavity for capturing an oral cavity image of a user, and generating a first reference line of the first oral cavity image; S102, using image segmentation technology to segment the first oral cavity image into a plurality of unit images, each unit image including a tooth and its associated gum image information, and generating a coding group for each unit image according to a relative position relationship between each unit image and a first reference line in the first oral cavity image and a preset coding rule; S103, obtaining a target oral cavity image acquired by the oral cavity image acquisition device under each target state parameter, and determining a target baseline of the target oral cavity image based on each target oral cavity image and its corresponding target state parameter; S104, based on each target oral cavity image and its target baseline, executing step S102 to generate all unit images of each target oral cavity image and their coding groups; S105, based on the unit images generated by the first oral cavity image and each target oral cavity image and their coding groups, all unit images with the same coding group are fused using an image fusion technology to obtain a panoramic unit image corresponding to each coding group; S106, based on all panoramic unit images, according to a preset oral morphology analysis algorithm and oral appearance analysis algorithm, obtain the user's oral morphology index and oral appearance index, perform weighted summation, and generate an oral comprehensive index; The panoramic unit images are all annotated with tooth types according to their coding groups, and the tooth types include incisors, canines, premolars, and molars; The method of obtaining the user's oral morphology index according to a preset oral morphology analysis algorithm specifically includes: S201, arranging the positions of the panoramic unit images according to their coding groups, and splicing the two adjacent panoramic unit images by using image fusion registration technology to generate a panoramic oral image; S202, based on the panoramic oral image, extract the first offset sequence and the second arrangement curve of the teeth, input them into the pre-trained tooth protrusion recognition model, and output the tooth protrusion posture grade value and the tooth protrusion posture label; the tooth protrusion posture label includes: normal, protrusion, and intrusion; S203, outputting the oral morphology index of the user according to the dental protrusion posture level value of the panoramic oral image; The oral appearance analysis algorithm is used to pre-train the oral recognition model corresponding to each oral detection target, including a feature extraction model and a classification model. The output end of the feature extraction model is connected to the input end of the classification model, specifically including: S301, attaching corresponding tooth type labels to each panoramic unit image, respectively inputting them into the oral cavity recognition models corresponding to each oral cavity detection target that has been pre-trained, and outputting the grade value of the panoramic unit image on each oral cavity detection target; S302, calculating the user's oral appearance index according to the following formula: Where P is the user’s oral appearance index, n is the number of the user’s panoramic unit images, m is the total number of oral detection targets, and w is the ij is the preset weight coefficient value of the ith panoramic unit image on the jth oral detection target, which is used to comprehensively reflect the importance of the oral detection target and the tooth type of the coding group of the panoramic unit image in the oral cavity to the user's oral appearance. ij is the level value of the i-th panoramic unit image on the j-th oral detection target.

2. The intelligent analysis method of oral images according to claim 1, characterized in that: The oral collection device is configured as a rotatable shooting rod, the end of which is equipped with a high-definition camera, a miniature fill light, and a position sensor; The position sensor is used to locate the target position and generate corresponding state parameters, which include the rotation angles of the camera in the horizontal and vertical directions; The target position is set as: a preset distance in a direction perpendicular to the straight line where the center seam of the lower front teeth is located in the oral cavity; The initial state parameters of the oral image acquisition device are: the camera focus point is facing the center gap of the lower front teeth, and the rotation angle is 0; the first reference line of the first oral image is set as: the camera focus point is positioned to the straight line in the vertical direction of the first oral image.

3. The intelligent analysis method of oral images according to claim 2, characterized in that: The preset coding rules are specifically: A1. Establish a coordinate system with the first reference line as the y-axis and the dividing line between the upper oral cavity and the lower oral cavity as the x-axis; A2. Traverse all unit images along the positive and negative directions of the x-axis and the positive and negative directions of the y-axis respectively, assign coding values ​​of the unit images on the x-axis and y-axis in sequence according to increasing or decreasing serial numbers, and generate a coding group (x, y) of the unit images.

4. The intelligent analysis method of oral images according to claim 2, characterized in that: In S103, based on each target oral cavity image, a target baseline of the target oral cavity image is determined according to its corresponding target state parameter, including: B1. Obtain a straight line in the vertical direction from the focus point of the camera to the target oral cavity image when shooting under the target state parameters, as the initial reference line; B2. Obtain the rotation angle in the horizontal direction of the target state parameter, invert it to obtain the target angle, rotate the initial reference line by the target angle, and obtain the target reference line of the target oral cavity image.

5. The intelligent analysis method of oral images according to claim 1, characterized in that: In S202, based on the panoramic oral image, extracting the first offset matrix and the second arrangement curve of the teeth specifically includes: S401, determining the center point of all panoramic unit images in the panoramic oral image; S403, according to a preset direction, respectively obtain the rays where the center points of two adjacent panoramic unit images are connected, and sequentially obtain the angle values ​​formed by the two adjacent rays until all panoramic unit images are traversed to obtain the upper tooth offset sequence and the lower tooth offset sequence; S404, combining the upper tooth offset sequence and the lower tooth offset sequence to obtain a first offset matrix; S405, acquiring a gum image in the panoramic oral image, and generating a gum simulation curve according to the gum image; S406, obtaining a dental arch image in the panoramic oral image, performing a texture analysis technique on the dental arch image, extracting texture information from the dental arch image, and generating a tooth arrangement curve using a curve connection technique; S407, combining the gum simulation curve and the tooth arrangement curve to form a second arrangement curve.

6. The intelligent analysis method of oral images according to claim 1, characterized in that: The oral morphology analysis algorithm further includes: S501, based on the panoramic oral image, using image segmentation technology to segment the panoramic oral image into a plurality of associated tooth groups, where the associated tooth group includes two adjacent tooth images; S502, based on each associated tooth group, using feature matching technology, identifying a target feature point group of each tooth image, where the target feature point group includes any of the tips, edge intersections, and inflection points of the tooth image; S503, based on the target feature point group of each tooth image, obtain a line segment connecting any two target feature points therein, take the line segment with the largest span on the tooth image as the first major axis of the tooth image, and obtain the angle value between the first major axis and the straight line where the extension direction of the tooth image is located; S504, determining whether the angle value is greater than the deviation threshold, if not, determining the first long axis as the target long axis of the tooth image, if yes, repeating step S503 to re-determine other long axes different from the first long axis, and determining the angle value until the angle value is less than the deviation threshold; S505, determining the target long axis of each tooth image in the associated tooth group, and calculating the torsion angle value of the associated tooth group according to the following formula: in, and denote the target long axis vector of each tooth image, respectively, and β is the torsion angle value of the associated tooth group; S506, according to the torsion angle values ​​of all associated tooth groups, the comprehensive torsion angle value of the panoramic oral image is calculated according to the following formula: Among them, β' is the comprehensive torsion angle value, K is the total number of associated tooth groups, β k is the torsion angle value of the kth associated tooth group, w k is the similarity of the tooth types of different tooth images in the kth associated tooth group; S507: According to the comprehensive torsion angle value of the panoramic oral image and in combination with a plurality of pre-set angle intervals, a tooth alignment grade value of the user is given.

7. The intelligent analysis method of oral images according to claim 6, characterized in that: In S203, outputting the oral morphology index of the user according to the dental protrusion posture level value of the panoramic oral image specifically includes: The tooth protrusion posture grade value and the tooth alignment grade value of the panoramic oral image are obtained, and weighted summation is performed to obtain the oral morphology index of the user.

8. The intelligent analysis method of oral images according to claim 6, characterized in that: In S501, the method further includes: S601, collecting a tooth gap image formed between two tooth images in an associated tooth group to obtain an associated tooth gap image; S602, extracting a first feature data group associated with the interdental image, the first feature data group including a color histogram, a shape descriptor, and a texture descriptor associated with the interdental image; S603, obtaining a second feature data set of a pre-prepared dental calculus sample image, the second feature data set including a color histogram, a shape descriptor, and a texture descriptor of the dental calculus sample image; S604, traversing and comparing the first feature data group and the second feature data group, and determining the connected domain area in the associated interdental image whose similarity reaches the similarity standard as the suspected calculus area; S605, inputting the suspected calculus area into the pre-trained calculus recognition model, outputting a probability distribution map of calculus, obtaining an area with a probability value greater than 90% and forming a connected domain in the probability distribution map, and determining it as a calculus image, otherwise, there is no calculus in the associated interdental image; S606, using image registration and stitching technology, determine the tooth image that matches the tartar image according to the position and shape of the tartar image in the associated interdental image, and stitch the tartar image to the corresponding position of the matching tooth image to update the associated tooth group.

Images