A multi-modal oral image intelligent analysis method, system, medium and product

Abstract

This application relates to the field of image data analysis technology, and in particular to a multimodal oral image intelligent analysis method, system, medium, and product. The method includes receiving multimodal self-captured oral images uploaded by patients through a client, including intraoral dentition photographs, facial soft tissue profile photographs, and dynamic occlusal video frames; identifying a set of crown contour feature points in the multimodal self-captured oral images; preprocessing the multimodal self-captured oral images based on the crown contour feature point set; identifying key feature points in the preprocessed multimodal self-captured oral images; and outputting hierarchical assessment results based on the identified key feature points and a pre-set oral health preliminary assessment model, generating a visualized assessment report, and pushing the visualized assessment report to the client and doctor's terminal. This application facilitates improved image processing accuracy, thereby meeting the application needs of subsequent health assessments.

Description

Technical Field

[0001] This application relates to the field of image data analysis technology, and in particular to a multimodal intelligent analysis method, system, medium, and product for oral images. Background Technology

[0002] With the rising incidence of oral diseases and the widespread adoption of digital healthcare technologies, patients' demand for convenient oral health assessments has increased significantly. Oral imaging analysis technology is gradually shifting from reliance on specialized in-hospital equipment to self-collection and remote diagnosis outside of hospitals. Traditional oral imaging relies on specialized equipment such as CBCT and intraoral scanners, which are complex and costly, and can only be performed in medical institutions. In recent years, with the significant increase in patients' demand for convenient oral health assessments, it has become a trend for patients to directly conduct health assessments by taking their own oral images.

[0003] However, when patients take their own photos, they may not be able to guarantee that the shooting angle is perpendicular to the dentition and the shooting distance is constant. This can easily lead to problems such as perspective distortion (memory to farthest point is smaller), rotational shift, and uneven scaling. Therefore, such images may have problems such as uneven resolution, severe perspective distortion, and blurred crown edges. Conventional techniques often use global geometric correction algorithms or general methods such as uniform filtering and noise reduction to process the oral images taken by patients before performing health assessments. However, there is a lack of differentiated correction schemes for self-taken images when processing oral images. Therefore, the processed oral images may still have problems such as local feature distortion, blurred edges of key tooth positions, and large deviations in feature point positioning, which cannot meet the accuracy requirements of subsequent health assessments. Summary of the Invention

[0004] To improve the accuracy of image processing and meet the application needs of subsequent health assessments, this application provides a multimodal oral image intelligent analysis method, system, medium, and product.

[0005] Firstly, this application provides a multimodal intelligent analysis method for oral images, employing the following technical solution: A multimodal intelligent analysis method for oral images includes: The system receives multimodal self-captured oral images uploaded by patients through a client. These multimodal self-captured oral images include intraoral dental arch photos, facial soft tissue profile photos, and dynamic occlusal video frames. The system identifies a set of crown contour feature points in the multimodal autonomously captured oral images, preprocesses the multimodal autonomously captured oral images based on the set of crown contour feature points, and identifies key feature points in the preprocessed multimodal autonomously captured oral images. Based on the identified key feature points and the preset oral health preliminary assessment model, hierarchical assessment results are output, a visual assessment report is generated, and the visual assessment report is pushed to the client and doctor's terminal.

[0006] By adopting the above technical solution, and through the identification and analysis of multimodal autonomously captured oral images, it is possible to achieve comprehensive oral information collection encompassing static morphology, dynamic function, and facial correlation. Complementary verification using multi-dimensional data effectively addresses the pain points of single-modal image information being incomplete and susceptible to interference from imaging defects, thereby improving the accuracy of remote diagnosis and health assessment. Using the crown contour feature point set as a coordinate reference and anatomical constraint, differentiated preprocessing is performed to address defects such as perspective distortion, uneven resolution, and blurred edges in autonomously captured images. This effectively eliminates interference from non-crown areas and corrects perspective distortion and blurred edges in autonomously captured images. Like inherent defects, key feature points with precise localization and high anatomical correlation are extracted and purified, which facilitates the provision of a high-quality data foundation for subsequent oral health assessments. Based on the quantitative indicators of key feature points, the preliminary oral health assessment model outputs hierarchical assessment results, which are simultaneously pushed to the patient's client and the doctor's terminal. For patients, they can clearly know the location, severity and corresponding suggestions of their oral problems, solving the pain point of not being able to understand professional imaging reports, and improving their understanding and cooperation with subsequent treatment plans. For doctors, the visualized report facilitates the rapid presentation of core diagnostic evidence, shortens the time for manual image interpretation and improves the efficiency of remote diagnosis.

[0007] In one possible implementation, identifying the set of crown contour feature points in the multimodal autonomously captured oral image includes: The pixel features of the multimodal autonomously captured oral images are identified, and edge detection is performed on the multimodal autonomously captured oral images based on the pixel features. The candidate crown edge information and the candidate anatomical position of each candidate feature point are extracted. The crown edge information includes contour line information and gradient direction. From the standard orthodontic intraoral image database, a set of standard feature points matching the patient's dentition morphology is selected. From the set of standard feature points, the standard feature points corresponding to each candidate feature point are determined. The candidate crown margin information of each candidate feature point is compared with the standard crown margin information of the corresponding standard feature point to determine the candidate confidence of each candidate feature point. The crown contour feature point set is determined based on the candidate confidence, candidate anatomical location, and candidate crown margin information of each candidate feature point.

[0008] By adopting the above technical solution, edge detection is carried out by identifying pixel features of multimodal autonomously captured oral images. Simultaneously, the outline, gradient direction, and other crown edge information and candidate anatomical positions of candidate feature points are extracted. This overcomes the limitations of conventional edge detection, which only extracts the outline and ignores anatomical correlation. It improves the correlation between candidate feature points and the real anatomical structure of the crown from the source and reduces the interference of false feature points. By introducing a standard orthodontic intraoral image database, a standard feature point set matching the patient's dentition morphology is selected for comparison. The abstract feature matching degree is transformed into a quantifiable candidate confidence level, which directly reflects the degree of conformity between candidate feature points and the real anatomical position. This provides an objective quantitative basis for subsequent feature point selection and improves the reliability of the feature point set.

[0009] In one possible implementation, determining the crown contour feature point set based on the candidate confidence score, candidate anatomical location, and candidate crown margin information for each candidate feature point includes: Multiple initial tooth positions are determined based on the candidate anatomical location of each candidate feature point. The mean confidence score of the initial candidate tooth position is identified for each initial tooth position. The initial tooth positions with the mean confidence score of the initial candidate tooth position that is higher than the corresponding tooth position confidence threshold are identified as key tooth positions. Based on the confidence difference between the initial candidate confidence mean of non-critical tooth positions and the corresponding tooth position confidence threshold, the supplementary analysis range of each non-critical tooth position is determined, and the adjacent candidate feature points of each non-critical tooth position are determined from the corresponding supplementary analysis range. Based on the adjacent parameters of the adjacent candidate feature points corresponding to each non-critical tooth position, the replacement value of each non-critical tooth position is determined, and non-critical tooth positions with replacement values ​​higher than the preset replacement threshold are identified as replacement tooth positions. Based on the key feature points corresponding to the key tooth positions, the candidate crown edge information of each key feature point, and the replacement feature points corresponding to the replacement tooth positions, the candidate crown edge information of each replacement feature point, a set of crown contour feature points is determined.

[0010] By adopting the above technical solution, through the hierarchical screening logic of prioritizing key tooth positions, evaluating the replacement of non-key tooth positions, and integrating multi-dimensional features, the problem of missed detection and unbalanced distribution of feature points caused by occlusion and distortion in self-captured images is solved. This facilitates the high integrity, high reliability, and reasonable spatial distribution of the crown contour feature point set, providing stable core data anchors for subsequent differentiated preprocessing and health assessment.

[0011] In one possible implementation, the filler value for a non-critical tooth position is determined based on the neighboring parameters of adjacent candidate feature points corresponding to the non-critical tooth position, including: Based on the adjacent parameters of each adjacent candidate feature point, the adjacent candidate anatomical position of each adjacent candidate feature point is determined, and multiple adjacent feature point combinations and the adjacent feature area corresponding to each adjacent feature point combination are determined based on each adjacent candidate anatomical position. The initial tooth positions of the adjacent candidate feature points included in the adjacent feature point combinations are different. The mean confidence value of each adjacent candidate is determined based on the adjacent candidate confidence value of each adjacent feature point in each adjacent feature point combination. Based on the adjacent feature area and the adjacent candidate confidence mean of each adjacent feature point combination, the initial compensation value corresponding to each adjacent feature point combination is determined, and the highest initial compensation value is determined as the compensation value of the non-critical tooth position.

[0012] By adopting the above technical solution, adjacent feature point combinations for different initial tooth positions are constructed based on adjacent candidate anatomical locations. At the same time, the adjacent feature area corresponding to the combination is calculated to limit the replacement range in terms of spatial distribution, avoid the one-sided interference of a single tooth position feature point, and prevent excessive spatial offset between the replacement feature point and non-critical tooth positions. This ensures that the spatial distribution uniformity of the feature point set after replacement is not destroyed. For each adjacent feature point combination, the confidence mean of adjacent candidates is calculated, expanding the confidence of a single feature point to an overall credibility assessment at the combination level. This reduces the impact of a single high-confidence pseudo-feature point on the replacement result. The initial replacement value is calculated by combining the adjacent feature area and the confidence mean of adjacent candidates. The two core requirements of spatial proximity and recognition reliability are quantified into a unified evaluation index, avoiding the limitations of single-dimensional evaluation, thereby facilitating the improvement of the accuracy of determining the replacement value corresponding to non-critical tooth positions.

[0013] In one possible implementation, the preprocessing of the multimodal autonomously captured oral images based on the set of crown contour feature points includes: The outward expansion size is determined based on the resolution characteristics of the multimodal autonomously captured images, the anatomical size of the crown, and the algorithm accuracy requirements. The core region of the crown is determined based on the crown contour feature point set and the outward expansion size. Based on the candidate anatomical position of each candidate feature point in the crown contour feature point set, the core region of the crown is divided into the anterior tooth region and the posterior tooth region; Multiple control feature points are identified from the core region of the crown, and differential fluoroscopic distortion correction is performed on the core region of the crown based on the anatomical location of each control feature point.

[0014] By adopting the above technical solution, the outward expansion size is determined by combining the resolution characteristics of the self-captured images, the anatomical size of the crown, and the accuracy requirements of the algorithm. Then, the core area of ​​the crown is anchored based on the crown contour feature point set, and the effective area containing the complete crown anatomical structure is accurately selected. At the same time, non-crown interference pixels are filtered to reduce the computational consumption of invalid areas for subsequent processing. The core area of ​​the crown is divided into anterior and posterior regions according to the candidate anatomical position of the feature points, avoiding the problems of insufficient correction in the anterior region and loss of details in the posterior region caused by global processing. This ensures the image quality of the entire dentition area is balanced. Control feature points within the core area are selected, and a targeted distortion correction model is constructed based on their anatomical position. This facilitates differentiated distortion correction in the anterior and posterior regions, thereby improving the accuracy of feature point positioning and providing a reliable guarantee for the quantitative analysis of subsequent health assessments.

[0015] In one possible implementation, the differential fluoroscopic distortion correction of the crown core region based on the anatomical location of each control feature point includes: The source coordinate set is determined based on the anatomical location of each control feature point; From the standard orthodontic intraoral image database, template images matching the patient's dentition morphology are selected, and the coordinates of the standard points corresponding to each control feature point are extracted and denoted as the target coordinate set. The preset homography transformation matrix is ​​optimized based on the source coordinate set and the target coordinate set; the core feature points corresponding to the core region of the crown are mapped based on the homography transformation matrix to perform differential distortion correction.

[0016] By adopting the above technical solution, the source coordinate set is determined based on the anatomical position of the control feature points, and the corresponding target coordinate set is matched from the standard orthodontic image library. This makes it easier to anchor the coordinate mapping relationship of distortion correction to the actual anatomical position of the crown, thereby facilitating the realization of differentiated fluoroscopic distortion correction in the core area of ​​the crown and improving the reliability of distortion correction.

[0017] Secondly, this application provides an analysis system, which adopts the following technical solution: An analysis system comprising: At least one processor; Memory; At least one application, wherein the at least one application is stored in memory and configured to be executed by at least one processor, the at least one application being configured to: execute the above-described multimodal oral image intelligent analysis method.

[0018] Thirdly, this application provides a computer-readable storage medium, which adopts the following technical solution: A computer-readable storage medium includes: a computer program stored thereon that can be loaded by a processor and executed by the above-described multimodal oral image intelligent analysis method.

[0019] Fourthly, this application provides a computer program product, which adopts the following technical solution: A computer program product includes a computer program that, when executed by a processor, implements the above-described intelligent analysis method for multimodal oral images.

[0020] In summary, this application includes at least one of the following beneficial technical effects: By recognizing and analyzing multimodal autonomously captured oral images, comprehensive oral information acquisition encompassing static morphology, dynamic function, and facial correlation can be achieved. Complementary verification using multi-dimensional data addresses the limitations of single-modal images, which are often incomplete and susceptible to interference from imaging defects. This improves the accuracy of remote diagnosis and health assessment. Using crown contour feature point sets as coordinate references and anatomical constraints, differentiated preprocessing is performed to address defects in autonomously captured images, such as perspective distortion, uneven resolution, and blurred edges. This effectively eliminates interference from non-crown areas and corrects inherent defects in autonomously captured images, including perspective distortion and blurred edges. Simultaneously, key feature points with precise localization and high anatomical correlation are extracted, facilitating the provision of a high-quality data foundation for subsequent oral health assessments. Based on the quantitative indicators of key feature points, a preliminary oral health assessment model outputs hierarchical assessment results, which are simultaneously pushed to patient clients and doctor terminals. For patients, this allows them to clearly understand the location, severity, and corresponding recommendations for their oral problems, resolving the pain point of not being able to understand professional imaging reports and improving their understanding and cooperation with subsequent treatment plans. For doctors, the visualized reports facilitate the rapid presentation of core diagnostic evidence, shortening manual image interpretation time and improving the efficiency of remote diagnosis.

[0021] By employing a hierarchical screening logic that prioritizes key tooth positions, assesses and supplements non-key tooth positions, and integrates multi-dimensional features, the system addresses the issues of missed feature point detection and imbalanced distribution caused by occlusion and distortion in self-captured images. This ensures high integrity, high reliability, and reasonable spatial distribution of the crown contour feature point set, providing stable core data anchors for subsequent differentiated preprocessing and health assessment. Attached Figure Description

[0022] Figure 1 This is a flowchart illustrating a multimodal intelligent oral imaging analysis method according to an embodiment of this application; Figure 2 This is a flowchart illustrating a method for determining a set of feature points on the crown contour in an embodiment of this application. Figure 3 This is a schematic diagram of the structure of an analysis system according to an embodiment of this application. Detailed Implementation

[0023] The following is in conjunction with the appendix Figures 1 to 3 This application will be described in further detail.

[0024] After reading this specification, those skilled in the art may make modifications to this embodiment without contributing any inventive step, but such modifications are protected by patent law as long as they fall within the scope of the claims of this application.

[0025] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] It should be noted that, in the optional embodiments of this application, the data related to object information, when applied to specific products or technologies, requires the permission or consent of the object. Furthermore, the collection, use, and processing of this data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. In other words, if the embodiments of this application involve data related to an object, it must be obtained with the object's authorization and consent, the authorization and consent of relevant departments, and in accordance with the relevant laws, regulations, and standards of the country and region. If the embodiments involve personal information, the acquisition of all personal information requires the individual's consent. If sensitive information is involved, the separate consent of the information subject is required. The embodiments also need to be implemented with the object's authorization and consent.

[0027] Specifically, this application provides a multimodal intelligent analysis method for oral images, executed by an analysis system. This analysis system can be a server or a terminal device. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal device can be a smartphone, tablet, laptop, desktop computer, etc., but is not limited to these. The terminal device and the server can be directly or indirectly connected via wired or wireless communication, and this application does not impose any limitations on this connection.

[0028] refer to Figure 1 , Figure 1 This is a flowchart illustrating a multimodal intelligent oral imaging analysis method according to an embodiment of this application. The method includes steps S110-S130, wherein: Step S110: Receive multimodal self-captured oral images uploaded by the patient through the client. The multimodal self-captured oral images include intraoral dental arch photos, facial soft tissue profile photos, and dynamic occlusal video frames.

[0029] Specifically, the analysis system can communicate with both the client and the doctor's end. Patients can log in to the client and upload self-taken images to the analysis system. Relevant medical staff can log in to the doctor's end to view the multimodal self-taken oral images uploaded by the patient, so as to remotely complete the preliminary diagnosis of oral health, risk classification and treatment recommendations. At the same time, it supports two-way interaction between doctors and patients, forming a closed-loop service process of patient self-collection, system intelligent analysis, doctor professional diagnosis and doctor-patient collaborative decision-making. The images captured by the patient include, but are not limited to, intraoral dental arch photographs, facial soft tissue profile photographs, and dynamic occlusal video frames. The shooting requirements for different types of images may vary. For example, intraoral dental arch photographs require the patient to point the camera at the maxillary anterior teeth, keeping the shooting device parallel to the dental arch and about 5-8 cm away from the mouth; facial soft tissue profile photographs require the patient to lean against a wall to maintain a stable posture, taking 1-2 left-side and 1-2 right-side profile photographs; dynamic occlusal video frames require the patient to naturally close their teeth, perform 3 slow chewing movements, and then hold the occlusal position for 5 seconds. To improve the reliability of multimodal autonomous oral imaging, voice prompts can be provided when the patient is taking the images. The specific content of the voice prompts is not specifically limited in this embodiment.

[0030] Step S120: Identify the crown contour feature point set in the multimodal autonomously captured oral images, preprocess the multimodal autonomously captured oral images based on the crown contour feature point set, and identify key feature points in the preprocessed multimodal autonomously captured oral images.

[0031] Specifically, because patients may not be able to ensure the imaging device is perpendicular to the dentition when holding the camera, fluoroscopic distortions such as stretching of the anterior teeth and compression of the posterior teeth are likely to occur. Additionally, due to the significant differences in brightness within the oral cavity (brighter anterior teeth, darker posterior teeth), the flash of the imaging device can easily cause reflections on the crown surface, creating false feature points. Therefore, the analysis system needs to preprocess the multimodal autonomously captured oral images sent by the client before performing subsequent evaluation operations. The method provided in this application does not directly perform conventional preprocessing methods such as noise reduction, deduplication, and resolution adjustment on the multimodal autonomously captured oral images. Instead, it first identifies the crown contour feature point set in the multimodal autonomously captured oral images, and then performs differential preprocessing on the multimodal autonomously captured oral images based on the crown contour feature point set. The specific process of identifying the crown contour feature point set in the multimodal autonomously captured oral images may include: The system identifies pixel features of multimodal autonomously captured oral images and performs edge detection on these images based on these pixel features. It extracts candidate crown edge information and candidate anatomical location for each candidate feature point. The crown edge information includes contour line information and gradient direction. From a standard orthodontic intraoral image database, a set of standard feature points matching the patient's dentition morphology is selected. Standard feature points corresponding to each candidate feature point are determined from this set. The candidate crown edge information of each candidate feature point is compared with the standard crown edge information of the corresponding standard feature point to determine the candidate confidence level of each candidate feature point. Based on the candidate confidence level, candidate anatomical location, and candidate crown edge information of each candidate feature point, a set of crown contour feature points is determined.

[0032] Specifically, the color self-captured oral images are first converted to grayscale images, and Gaussian filtering is used to eliminate shooting noise, so as to unify the brightness dimension of pixels and avoid the interference of RGB three-channel color threshold calculation. Based on the preset feature recognition algorithm, the pixel features corresponding to the multimodal self-captured oral images are identified. Then, based on the pixel features and the maximum inter-class variance method, the multimodal self-captured oral images are divided into multiple local sub-blocks, and the histogram segmentation threshold corresponding to each local sub-block is calculated to generate a binary mask of the dentition region. That is, this scheme does not calculate a global threshold for the entire grayscale image, but divides the grayscale image into several local sub-blocks (such as 32×32 pixel sub-regions), calculates the histogram and inter-class variance of each sub-block, and then generates an adaptive local threshold matrix. The core purpose of this is to adapt to the uneven lighting characteristics of self-captured oral images (such as large differences in brightness inside the oral cavity) and avoid the failure of tooth crown edge region segmentation due to the global threshold. The pixel values ​​of the binary mask only contain two states: 0 and 1. A pixel value of 1 indicates that the location belongs to the target area of ​​the dentition (such as the effective pixels of the crown and interdental spaces); a pixel value of 0 indicates that the location belongs to the background interference area (such as non-dental pixels like the lips, tongue, gums, and the shooting background). The method for determining the binary mask of the dentition area is not specifically limited in this embodiment. Since the crown edge may be blurred due to occlusion or insufficient lighting during the patient's self-portrait, the initial contour of the feature point fitting may have slight gaps (such as the gap between two teeth not closing or the crown edge being locally broken). Direct use may cause some crown areas to be missed during subsequent preprocessing. Therefore, after determining the binary mask of the dentition area, a dilation-erosion morphological operation can be performed to obtain the complete coarse positioning area of ​​the dentition. The core purpose is to repair the defects of the initial mask, optimize the dentition contour morphology, and improve the reliability of subsequent accurate positioning. Finally, edge detection is performed on the complete coarse positioning area of ​​the dentition based on a preset edge detection algorithm to obtain the candidate crown edge information and the candidate anatomical position of each candidate feature point. Candidate feature points are pixels in multimodal autonomously captured oral images. A tooth position may correspond to multiple candidate feature points. Candidate feature points are not the final precise crown feature points, but rather pixels with crown edge feature attributes extracted from multimodal autonomously captured oral images based on pixel features and edge detection results.

[0033] First, the pixel coordinates of all candidate feature points can be marked in the coarse localization area of ​​the complete dentition. Starting from a single candidate feature point, 8-neighborhood connectivity analysis (or 4-neighborhood, adjusted according to image resolution) is used to traverse the edge pixels with a pixel value of 1 around it. These pixels are connected sequentially to form a local continuous contour segment with the candidate feature point as the node. Then, the contour curvature (e.g., the contour curvature is large at the cusp position and small at the cervical margin position), contour length (e.g., the occlusal surface contour length of a single crown and the lateral surface contour length of a crown), and contour continuity (e.g., whether it is a complete closed single crown contour or a broken edge segment) of the corresponding local continuous contour segment are identified. The contour curvature, contour length, and contour continuity are determined as contour information. Within the coarse localization area of ​​the complete dentition, the gradient direction of a candidate feature point refers to the direction in which the pixel gray value changes most drastically. It can be determined by calculating the gray-level gradient components of the candidate feature point based on the Sobel operator.

[0034] By combining prior knowledge of dental anatomy (such as "the edge contour of a single tooth crown is a closed curve" and "the contour line of the anterior tooth region has a large curvature, while that of the posterior tooth region is relatively flat"), the anatomical location of the tooth crown corresponding to the contour line of the candidate feature point can be determined. For example, if the contour line is a closed curve and its length matches the pixel size of a single anterior tooth crown, it is labeled as "complete contour line of a single anterior tooth crown"; if the contour line is a non-closed arc segment and is located at the top of the tooth crown, it is labeled as "local contour line of the cusp region".

[0035] The standard orthodontic intraoral imaging database contains multiple standardized dentition images. Each dentition image is labeled with multiple standard feature points. Each standard feature point corresponds to unique standard crown margin information and standard anatomical position. The dentition morphology of the patient can be identified first from the coarse localization area of ​​the patient's complete dentition. Then, a set of standard feature points matching the dentition morphology of the patient is selected from the standard orthodontic intraoral imaging database. The candidate crown margin information of each candidate feature point is compared with the standard crown margin information of the corresponding standard feature point. The similarity of the margin information of each candidate feature point with each standard feature point in the corresponding standard feature point subset is calculated, which is the candidate confidence score. The confidence score is a quantitative score of 0-100% used to characterize the probability that the candidate feature point identified by the algorithm "belongs to the real crown anatomical feature point". Finally, based on the candidate confidence score, candidate anatomical position and candidate crown margin information of each candidate feature point, the crown contour feature point set is determined from all candidate feature points to ensure that the feature points in the crown contour feature point set are evenly distributed.

[0036] After determining the set of crown contour feature points in the multimodal autonomously captured oral images, differential correction processing can be performed on the multimodal autonomously captured oral images based on the crown contour feature point set. Specifically, this can include: The outward expansion dimension is determined based on the resolution characteristics of the multimodal autonomously captured images, the anatomical size of the crown, and the accuracy requirements of the algorithm. The core region of the crown is determined based on the crown contour feature point set and the outward expansion dimension. The core region of the crown is divided into anterior and posterior regions based on the candidate anatomical position of each candidate feature point in the crown contour feature point set. Multiple control feature points are determined from the core region of the crown, and differential perspective distortion correction is performed on the core region of the crown based on the anatomical position of each control feature point.

[0037] Specifically, the method provided in this application uses a set of crown contour feature points as anchor points to focus correction on key teeth, achieving targeted preprocessing. First, the multimodal self-captured oral images uploaded by the patient can be identified to determine the corresponding resolution characteristics. Resolution characteristics include, but are not limited to, pixel resolution (directly determining the image's pixel density), pixel equivalent (a crucial bridge connecting the pixel space and the physical anatomical space of the crown, directly affecting the rationality of pixel values ​​for outward expansion), and effective pixel ratio (which can assist in optimizing the algorithm's processing range and reducing computational consumption from invalid pixels). The anatomical dimensions of the crown are divided into the local dimensions of a single crown and the overall span of the dentition, which can be identified from the multimodal self-captured images based on a preset feature recognition algorithm. The algorithm's accuracy requirements can be set by relevant technical personnel according to actual needs; specific values ​​are not specifically limited in this application embodiment.

[0038] After determining the resolution characteristics, crown anatomical dimensions, and algorithm accuracy requirements, the outward expansion dimensions corresponding to the multimodal autonomously captured images can be determined based on a preset expansion dimension mapping relationship. The preset expansion dimension mapping relationship is the correspondence between the parameter combination of resolution characteristics, crown anatomical dimensions, and algorithm accuracy requirements and the outward expansion dimensions. The specific content of this correspondence relationship is not specifically limited in the embodiments of this application.

[0039] Based on the pixel coordinates of each candidate feature point in the crown contour feature point set, a minimum rectangular area covering all candidate feature points is determined as the reference range of the crown core region. According to the determined outward expansion size, the reference range is uniformly expanded in four directions to obtain the crown core region. Based on the candidate anatomical position of each candidate feature point and the preset orthodontic clinical tooth position zoning rules, the core region is subdivided into two sub-regions: the anterior region and the posterior region. Subsequently, multiple stable crown feature points are determined from the crown core region as control points, and their reference coordinates in standard orthodontic intraoral images are calculated. A homography transformation matrix constrained by the feature points is constructed. Perspective transformation is performed only on the crown core region to correct crown deformation caused by the tilt of the shooting angle. Different transformation weights are used for the anterior and posterior regions; for example, the weight of the anterior region (0.8) is higher than that of the posterior region (0.5), prioritizing the accuracy of the anterior region, which is the focus of initial orthodontic screening.

[0040] In addition to performing differential perspective distortion correction on the core area of ​​the crown, it can also perform feature point-constrained local histogram equalization on the core area of ​​the crown, and adaptively adjust for areas with insufficient or overexposed lighting. Specifically, it first calculates the local gray-scale mean of 10×10 pixels around the feature points in the core area of ​​the crown, then performs feature point-constrained local histogram equalization on the areas with insufficient lighting (local gray-scale mean < 50), enhancing the contrast only in the core area and avoiding overexposure of the background area. Finally, it fills the overexposed areas (local gray-scale mean > 200) using the median gray-scale value of the pixels around the feature points to repair the loss of details at the crown edge.

[0041] It can also perform high-intensity Gaussian filtering on the background area and remove occluders. Specifically, it can first generate a mask for the core area of ​​the crown based on the location of the feature point set, then perform high-intensity Gaussian filtering on the background area to suppress noise interference from occluders such as lips and tongue in multimodal autonomously captured images, and then use a semantic segmentation algorithm to directly remove occluders (such as fingers) in the background area to avoid their interference with feature point recognition.

[0042] When determining multiple control feature points from the core region of the crown, the process can specifically include: First, filtering multiple candidate feature points based on their confidence level to obtain a first candidate feature point group. Unstable feature points below a preset confidence threshold, or those caused by image distortion or local occlusion, are removed to ensure the final selected feature points meet the transformation matrix solution requirements. Next, the spatial coordinate variance of each candidate feature point in the first candidate feature point group is calculated to ensure that the number of candidate feature points in the anterior and posterior regions is roughly equal, and the pixel distance deviation between any two candidate feature points is ≤10%. If the distribution is uneven, the candidate feature points are readjusted to avoid the transformation matrix failing due to feature point concentration, resulting in a second candidate feature point group. Finally, the second candidate feature point group undergoes multi-frame consistency screening. If the patient uploaded multiple multimodal self-taken images of the same area, the coordinates of the candidate feature points in each image are extracted. The standard deviation of the coordinates of each candidate feature point in the second candidate feature point group is calculated. Only candidate feature points with a standard deviation ≤2 pixels are retained, and candidate feature points with large coordinate fluctuations are identified as unstable and replaced, resulting in the final multiple control feature points. Based on the anatomical location of each control feature point, matching perspective distortion correction parameters and strategies can be applied to the anterior and posterior regions of the core area of ​​the crown, respectively, to achieve differentiated distortion correction in the anterior and posterior regions, thereby improving the accuracy of feature point positioning.

[0043] Furthermore, to improve the accuracy of differential distortion correction results, differential fluoroscopic distortion correction is performed on the core area of ​​the crown based on the anatomical location of each control feature point. Specifically, this may include: The source coordinate set is determined based on the anatomical location of each control feature point; template images matching the patient's dentition morphology are selected from the standard orthodontic intraoral image database, and the coordinates of the standard points corresponding to each control feature point are extracted and denoted as the target coordinate set; the preset homography transformation matrix is ​​optimized based on the source coordinate set and the target coordinate set; the core feature points corresponding to the core area of ​​the crown are mapped based on the homography transformation matrix to perform differential distortion correction.

[0044] Specifically, based on the anatomical location of all control feature points, a set of source coordinates corresponding to all control feature points is determined. From a standard orthodontic intraoral image database, a template image matching the patient's dentition morphology is selected, and the standard coordinates of multiple corresponding standard feature points are extracted and integrated into a target coordinate set. A preset homography transformation matrix is ​​used to transform the coordinates of each control feature point in the source coordinate set to obtain a transformed source coordinate set. Then, the coordinate error between the transformed control coordinates of each control feature point in the transformed source coordinate set and the standard coordinates of the corresponding standard point in the target coordinate set is calculated. Subsequently, the average value of the coordinate error values ​​of all control feature points is calculated and used as a matrix optimization parameter. Finally, the preset homography transformation matrix is ​​optimized based on the matrix optimization parameter to obtain an optimized homography transformation matrix. The larger the matrix optimization parameter, the worse the fit between the preset homography transformation matrix and the current patient's dentition features. Based on the optimized homography transformation matrix, each control feature point is specifically corrected to maximize the restoration of the true anatomical morphology and spatial position relationship of the patient's crown core area.

[0045] Because multimodal autonomous oral imaging may have issues such as perspective distortion, angular deviation, and soft tissue occlusion, the original coordinates of crown feature points may deviate from their actual anatomical positions (e.g., the incisal edge is stretched and the cusp position is offset). The optimized homography transformation matrix, by integrating the anatomical prior constraints of the standard dentition and the matching error feedback of the patient's personalized feature points, facilitates targeted correction of multimodal autonomous oral imaging, making it conform to the anatomical arrangement of normal dentition, avoiding over-correction of non-core areas, and ensuring the accuracy of the restoration of the core crown morphology.

[0046] The above method can complete the fine preprocessing of multimodal autonomous oral images. At this time, based on the preset feature recognition algorithm, key feature points containing preset key features can be identified from the preprocessed multimodal autonomous oral images. The key feature points may be feature points of lesions or feature points in key oral regions. The specific preset key features are not specifically limited in this application embodiment. They can be extracted from the assessment needs provided by relevant patients or determined by relevant medical staff according to preset health assessment standards. For example, when the assessment need is "the gums often bleed when brushing teeth and there is also bleeding when cleaning between teeth with dental floss", the preset key features corresponding to the assessment need can be determined based on the preset semantic recognition algorithm. These can be the boundary points of the congested and reddened areas of the gingival margin and the inflection points of the gingival attachment lines.

[0047] Step S130: Based on the identified key feature points and the preset oral health preliminary assessment model, output hierarchical assessment results, generate a visual assessment report, and push the visual assessment report to the patient client and doctor terminal.

[0048] Specifically, the identified key feature points (such as the boundary of a carious cavity, the inflection point of gingival recession, and the center point of leukoplakia) can first undergo multidimensional parameter calibration to generate a feature quantification dataset. This dataset includes coordinate positions, feature dimensions, and associated parameter values ​​(determined by the relative distance between the key feature point and surrounding tissues; the closer the relative distance, the higher the associated parameter value). The health level corresponding to each key feature point is then determined according to a preset health level mapping rule. This preset health level mapping rule is the mapping relationship between the feature quantification dataset of the key feature point and the health level. Specific details are not limited in this embodiment. The health level is defined as healthy, low-risk, medium-risk, and high-risk. Finally, the hierarchical assessment results are output according to the preset output rules, which can be as follows: Health level: No abnormal feature points, that is, the parameters corresponding to the key feature points are all within the health threshold; Low risk: There are 1-2 low-risk feature points, no medium / high-risk feature points, and no obvious signs of lesions in local tissues; Medium risk: There is 1 medium-risk feature point, or ≥3 low-risk feature points, requiring further clinical examination; High risk: There is ≥1 high-risk feature point, or ≥2 medium-risk feature points, indicating that there may be a clear lesion, requiring immediate medical attention.

[0049] Image rendering technology can be used to accurately annotate preprocessed multimodal oral images based on a patient-based template (using different colored marker boxes to distinguish feature regions of different risk levels; adding text labels at feature point locations, etc.), resulting in a visual assessment report fed back to the client. The patient-based template includes annotated oral images, risk level explanations, and simplified health advice (e.g., "Low-risk demineralization exists in the left posterior teeth; it is recommended to reduce sweets intake and improve oral hygiene"). Similarly, image rendering technology can be used to accurately annotate preprocessed multimodal oral images based on a doctor-based template, resulting in a visual assessment report fed back to the doctor's terminal. The doctor-based template can add a feature point quantification parameter table, model evaluation confidence level, and links to the original image data, based on the patient-based template, to support further analysis by relevant medical personnel. Specific annotation methods, patient-based templates, and doctor-based templates are not specifically limited in this embodiment.

[0050] In this embodiment, by recognizing and analyzing multimodal autonomously captured oral images, it is convenient to achieve comprehensive oral information collection encompassing static morphology, dynamic function, and facial correlation. Through complementary verification of multi-dimensional data, it effectively addresses the pain points of single-modal image information being incomplete and susceptible to interference from imaging defects, thereby improving the accuracy of remote diagnosis and health assessment. Using the crown contour feature point set as a coordinate reference and anatomical constraint, differentiated preprocessing is performed to address defects such as perspective distortion, uneven resolution, and blurred edges in autonomously captured images. This effectively eliminates interference from non-crown areas and repairs perspective distortion and blurred edges in autonomously captured images. While possessing inherent flaws, the system extracts and refines key feature points with precise localization and high anatomical relevance, facilitating a high-quality data foundation for subsequent oral health assessments. Based on the quantitative indicators of these key feature points, a preliminary oral health assessment model is constructed, outputting hierarchical assessment results, which are simultaneously pushed to patient clients and doctor terminals. For patients, this allows them to clearly understand the location, severity, and corresponding recommendations for their oral problems, addressing the pain point of not being able to understand professional imaging reports and improving their understanding and cooperation with subsequent treatment plans. For doctors, the visualized reports facilitate the rapid presentation of core diagnostic evidence, shortening manual image interpretation time and improving the efficiency of remote diagnosis.

[0051] Furthermore, to improve the accuracy of determining the crown contour feature point set, the method provided in this application, when determining the crown contour feature point set based on the candidate confidence, candidate anatomical location, and candidate crown edge information of each candidate feature point, may specifically include steps S210-S240, such as... Figure 2 As shown, where: Step S210: Determine multiple initial tooth positions based on the candidate anatomical location of each candidate feature point, identify the mean initial candidate confidence score corresponding to each initial tooth position, and determine the initial tooth positions with the mean initial candidate confidence score higher than the corresponding tooth position confidence threshold as key tooth positions.

[0052] Specifically, based on the candidate anatomical position of each candidate feature point, all candidate feature points are first divided into multiple candidate groups. Each candidate group may contain one or more candidate feature points. The specific number is not specifically limited in this embodiment, as long as the candidate anatomical positions of all candidate feature points in the candidate group are located within the anatomical region of the same tooth position. The initial tooth position is the tooth position containing the candidate feature point group. Due to the limited conditions and accuracy of the patient's self-taken photos, there may be tooth positions without candidate groups due to visual obstruction, i.e., obstructed tooth positions. In other words, all initial tooth positions may not be all the patient's tooth positions.

[0053] After identifying the initial tooth position, the average candidate confidence score of all candidate feature points in the candidate group corresponding to the initial tooth position is calculated to obtain the average initial candidate confidence score for each initial tooth position. This average initial candidate confidence score is then compared with the tooth position confidence threshold corresponding to the initial tooth position. If the average initial candidate confidence score of the initial tooth position is higher than the corresponding tooth position confidence threshold, the initial tooth position is identified as a critical tooth position. Since different initial tooth positions present different imaging challenges, when identifying critical tooth positions, the corresponding tooth position confidence threshold can be determined based on the tooth position of the initial tooth position. The further the tooth position is from the center point of the frontal view, the lower the corresponding tooth position confidence threshold. This can be determined based on a preset confidence threshold mapping relationship, which is the correspondence between the tooth position distance between the tooth position and the center point of the frontal view and the tooth position confidence threshold. The specific details are not specifically limited in this embodiment.

[0054] Step S220: Based on the confidence difference between the initial candidate confidence mean of non-critical tooth positions and the corresponding tooth position confidence threshold, determine the supplementary analysis range for each non-critical tooth position, and determine the adjacent candidate feature points of each non-critical tooth position from the corresponding supplementary analysis range.

[0055] Specifically, non-critical tooth positions refer to the tooth positions other than the critical tooth positions in the initial tooth positions. The average confidence score of the initial candidates for non-critical tooth positions is not higher than the corresponding tooth position confidence threshold. In this case, the non-critical tooth positions are not directly eliminated. Instead, the adjacent candidate feature points of the non-critical tooth positions are analyzed to determine whether the non-critical tooth positions need to be added to the final crown contour feature point set. For any non-critical tooth position, the confidence difference between the average confidence score of the initial candidates and the corresponding tooth position confidence threshold can be calculated first. Then, the supplementary analysis area corresponding to the non-critical tooth position is determined according to the preset supplementary analysis area mapping relationship. Based on the non-critical tooth position and the supplementary analysis area, the supplementary analysis range corresponding to the non-critical tooth position is determined. The supplementary analysis range is a circle centered on the non-critical tooth position, and the area of ​​the supplementary analysis range is the supplementary analysis area. The preset supplementary analysis area mapping relationship is the correspondence between the confidence difference and the supplementary analysis area. The smaller the confidence difference, the larger the corresponding supplementary analysis area. The specific details are not specifically limited in the embodiments of this application. The adjacent candidate feature points within the supplementary analysis range are those corresponding to the adjacent tooth positions of non-critical tooth positions and falling within the supplementary analysis range. Based on this method, the adjacent candidate feature points for each non-critical tooth position can be determined.

[0056] Step S230: Determine the replacement value of each non-critical tooth position based on the adjacent parameters of the adjacent candidate feature points corresponding to each non-critical tooth position, and identify non-critical tooth positions with replacement values ​​higher than the preset replacement threshold as replacement tooth positions.

[0057] Specifically, for any non-critical tooth position, the adjacent parameters of adjacent candidate feature points include, but are not limited to, the adjacent candidate anatomical location and the adjacent candidate confidence level. By analyzing the adjacent parameters of the adjacent candidate feature points corresponding to the non-critical tooth position, a quantitative analysis can be performed to determine whether the non-critical tooth position can be used as a replacement tooth position. Specifically, this can include: Based on the adjacent parameters of each adjacent candidate feature point, the adjacent candidate anatomical position of each adjacent candidate feature point is determined. Based on each adjacent candidate anatomical position, multiple adjacent feature point combinations and the adjacent feature area corresponding to each adjacent feature point combination are determined. The initial tooth positions corresponding to the adjacent candidate feature points included in the adjacent feature point combinations are different. The adjacent candidate confidence mean of each adjacent feature point combination is determined according to the adjacent candidate confidence of each adjacent feature point in each adjacent feature point combination. Based on the adjacent feature area and the adjacent candidate confidence mean of each adjacent feature point combination, the initial replacement value corresponding to each adjacent feature point combination is determined, and the highest initial replacement value is determined as the replacement value for non-critical tooth positions.

[0058] Specifically, since the adjacent candidate feature points corresponding to non-critical tooth positions originate from the supplementary analysis range corresponding to the non-critical tooth positions, and the supplementary analysis range is a circular area centered on the non-critical tooth position, it can be concluded that the adjacent candidate feature points corresponding to non-critical tooth positions originate from the adjacent tooth positions of the non-critical tooth positions. Based on the adjacent tooth position (initial tooth position) to which each adjacent candidate feature belongs, multiple adjacent candidate feature points are combined to obtain multiple adjacent feature point combinations. The adjacent tooth positions to which the adjacent candidate features belong in the adjacent feature point combinations are different. Based on the adjacent candidate anatomical position of each adjacent candidate feature point in the adjacent feature point combination, the adjacent feature area corresponding to the adjacent feature point combination is determined. At the same time, the adjacent candidate confidence mean corresponding to the adjacent feature point combination is calculated based on the adjacent candidate confidence of each adjacent feature point in each adjacent feature point combination. Finally, the adjacent feature area and the adjacent candidate confidence mean of each adjacent feature point combination are quantized and normalized to obtain the initial supplementary value corresponding to each adjacent feature point combination. The initial supplementary values ​​of all adjacent feature point combinations are compared, and the highest initial supplementary value is determined as the supplementary value for the non-critical tooth position.

[0059] For each combination of adjacent feature points, the confidence mean of adjacent candidates is calculated, expanding the confidence of a single feature point to an overall credibility assessment at the combination level. This reduces the impact of a single high-confidence pseudo-feature point on the replacement result. The initial replacement value is calculated by combining the area of ​​adjacent features and the confidence mean of adjacent candidates. The two core requirements of spatial proximity and recognition reliability are quantified into a unified evaluation index, avoiding the limitations of single-dimensional evaluation, thereby facilitating the improvement of the accuracy of determining the replacement value corresponding to non-critical tooth positions.

[0060] Based on the above method, the replacement value of each non-critical tooth position can be determined. At this time, the replacement value of each non-critical tooth position can be compared with the preset replacement threshold, and the non-critical tooth positions with replacement values ​​higher than the preset replacement threshold are determined as replacement tooth positions. In this embodiment of the application, the preset replacement threshold is not specifically limited, but can be set by relevant personnel according to actual needs.

[0061] Step S240: Based on the key feature points corresponding to the key tooth positions, the candidate crown edge information of each key feature point, and the replacement feature points corresponding to the replacement tooth positions, the candidate crown edge information of each replacement feature point, determine the crown contour feature point set.

[0062] Specifically, after determining the replacement tooth position, a crown contour feature point set can be constructed together with the key tooth position and the replacement tooth position. That is, the crown contour feature point set consists of the key feature points corresponding to the key tooth position, the candidate crown edge information of each key feature point, and the replacement feature points corresponding to the replacement tooth position, the candidate crown edge information of each replacement feature point. Through the hierarchical screening logic of prioritizing key tooth positions, evaluating replacement of non-key tooth positions, and integrating multi-dimensional features, the problem of missed feature point detection and distribution imbalance caused by occlusion and distortion in self-captured images is solved. This facilitates the high integrity, high reliability, and reasonable spatial distribution of the crown contour feature point set, providing a stable core data anchor for subsequent differentiated preprocessing and health assessment.

[0063] This application provides an analysis system, such as... Figure 3 As shown, Figure 3 The analysis system 300 shown includes a processor 301 and a memory 303. The processor 301 and the memory 303 are connected, for example, via a bus 302. Optionally, the analysis system 300 may also include a transceiver 304. It should be noted that in practical applications, the transceiver 304 is not limited to one type, and the structure of this analysis system 300 does not constitute a limitation on the embodiments of this application.

[0064] Processor 301 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 301 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0065] Bus 302 may include a pathway for transmitting information between the aforementioned components. Bus 302 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 302 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 3The symbol is represented by only one line, but this does not mean that there is only one bus or one type of bus.

[0066] The memory 303 may be a ROM (Read Only Memory) or other type of static storage device capable of storing static information and instructions, RAM (Random Access Memory) or other type of dynamic storage device capable of storing information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0067] The memory 303 is used to store application code that executes the solution of this application, and its execution is controlled by the processor 301. The processor 301 is used to execute the application code stored in the memory 303 to implement the content shown in the foregoing method embodiments.

[0068] The analysis system includes, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (such as in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. It can also include servers. Figure 3 The analysis system shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0069] This application provides a computer-readable storage medium storing a computer program that, when run on a computer, enables the computer to execute the corresponding content in the aforementioned method embodiments.

[0070] This application provides a computer program product including a computer program that, when executed by a processor, implements the methods described in any of the above embodiments.

[0071] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0072] The above description is only a partial embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A multimodal intelligent analysis method for oral images, characterized in that, include: The system receives multimodal self-captured oral images uploaded by patients through a client. These multimodal self-captured oral images include intraoral dental arch photos, facial soft tissue profile photos, and dynamic occlusal video frames. The system identifies a set of crown contour feature points in the multimodal autonomously captured oral images, preprocesses the multimodal autonomously captured oral images based on the set of crown contour feature points, and identifies key feature points in the preprocessed multimodal autonomously captured oral images. Based on the identified key feature points and the preset oral health preliminary assessment model, hierarchical assessment results are output, a visual assessment report is generated, and the visual assessment report is pushed to the client and doctor's terminal.

2. The multimodal oral imaging intelligent analysis method according to claim 1, characterized in that, The identification of the set of crown contour feature points in the multimodal autonomously captured oral images includes: The pixel features of the multimodal autonomously captured oral images are identified, and edge detection is performed on the multimodal autonomously captured oral images based on the pixel features. The candidate crown edge information and the candidate anatomical position of each candidate feature point are extracted. The crown edge information includes contour line information and gradient direction. From the standard orthodontic intraoral image database, a set of standard feature points matching the patient's dentition morphology is selected. From the set of standard feature points, the standard feature points corresponding to each candidate feature point are determined. The candidate crown margin information of each candidate feature point is compared with the standard crown margin information of the corresponding standard feature point to determine the candidate confidence of each candidate feature point. The crown contour feature point set is determined based on the candidate confidence, candidate anatomical location, and candidate crown margin information of each candidate feature point.

3. The multimodal oral imaging intelligent analysis method according to claim 2, characterized in that, The determination of the crown contour feature point set based on the candidate confidence score, candidate anatomical location, and candidate crown margin information of each candidate feature point includes: Multiple initial tooth positions are determined based on the candidate anatomical location of each candidate feature point. The mean confidence score of the initial candidate tooth position is identified for each initial tooth position. The initial tooth positions with the mean confidence score of the initial candidate tooth position that is higher than the corresponding tooth position confidence threshold are identified as key tooth positions. Based on the confidence difference between the initial candidate confidence mean of non-critical tooth positions and the corresponding tooth position confidence threshold, the supplementary analysis range of each non-critical tooth position is determined, and the adjacent candidate feature points of each non-critical tooth position are determined from the corresponding supplementary analysis range. Based on the adjacent parameters of the adjacent candidate feature points corresponding to each non-critical tooth position, the replacement value of each non-critical tooth position is determined, and non-critical tooth positions with replacement values ​​higher than the preset replacement threshold are identified as replacement tooth positions. Based on the key feature points corresponding to the key tooth positions, the candidate crown edge information of each key feature point, and the replacement feature points corresponding to the replacement tooth positions, the candidate crown edge information of each replacement feature point, a set of crown contour feature points is determined.

4. The multimodal intelligent analysis method for oral images according to claim 3, characterized in that, Based on the adjacent parameters of the adjacent candidate feature points corresponding to non-critical tooth positions, the replacement value of the non-critical tooth position is determined, including: Based on the adjacent parameters of each adjacent candidate feature point, the adjacent candidate anatomical position of each adjacent candidate feature point is determined, and multiple adjacent feature point combinations and the adjacent feature area corresponding to each adjacent feature point combination are determined based on each adjacent candidate anatomical position. The initial tooth positions of the adjacent candidate feature points included in the adjacent feature point combinations are different. The mean confidence value of each adjacent candidate is determined based on the adjacent candidate confidence value of each adjacent feature point in each adjacent feature point combination. Based on the adjacent feature area and the adjacent candidate confidence mean of each adjacent feature point combination, the initial compensation value corresponding to each adjacent feature point combination is determined, and the highest initial compensation value is determined as the compensation value of the non-critical tooth position.

5. The multimodal oral imaging intelligent analysis method according to claim 2, characterized in that, The preprocessing of the multimodal autonomously captured oral images based on the crown contour feature point set includes: The outward expansion size is determined based on the resolution characteristics of the multimodal autonomously captured images, the anatomical size of the crown, and the algorithm accuracy requirements. The core region of the crown is determined based on the crown contour feature point set and the outward expansion size. Based on the candidate anatomical position of each candidate feature point in the crown contour feature point set, the core region of the crown is divided into the anterior tooth region and the posterior tooth region; Multiple control feature points are identified from the core region of the crown, and differential fluoroscopic distortion correction is performed on the core region of the crown based on the anatomical location of each control feature point.

6. The multimodal oral imaging intelligent analysis method according to claim 5, characterized in that, The differential fluoroscopic distortion correction of the crown core region based on the anatomical location of each control feature point includes: The source coordinate set is determined based on the anatomical location of each control feature point; From the standard orthodontic intraoral image database, template images matching the patient's dentition morphology are selected, and the coordinates of the standard points corresponding to each control feature point are extracted and denoted as the target coordinate set. The preset homography transformation matrix is ​​optimized based on the source coordinate set and the target coordinate set; the core feature points corresponding to the core region of the crown are mapped based on the homography transformation matrix to perform differential distortion correction.

7. An analysis system, characterized in that, The analysis system includes: At least one processor; Memory; At least one application, wherein the at least one application is stored in memory and configured to be executed by at least one processor, the at least one application being configured to: perform a multimodal oral image intelligent analysis method according to any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, include: The computer program is stored and can be loaded by a processor and executed as described in any one of claims 1-6.

9. A computer program product, characterized in that, The method includes a computer program that, when executed by a processor, implements the steps of the multimodal oral image intelligent analysis method according to any one of claims 1-6.

Images