Abnormal tooth part identification method, model training method, device and equipment

Abstract

The present application provides a method for identifying tooth abnormal parts, a model training method, an apparatus, a device and a storage medium. The method for identifying tooth abnormal parts includes: projecting a first tooth model to be identified from a three-dimensional space to a two-dimensional space according to a plurality of different projection viewing angles to obtain a plurality of first tooth images; wherein the different projection viewing angles are used to make each tooth in the first tooth model have projection results under at least two projection viewing angles in the plurality of first tooth images; performing tooth abnormal part identification processing on the plurality of first tooth images using a pre-trained tooth abnormal part identification model to obtain abnormal part identification results corresponding to the plurality of first tooth images respectively; screening and processing the abnormal part identification results corresponding to the plurality of first tooth images respectively to determine the target abnormal part. The efficiency and accuracy of tooth abnormal part identification are improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method for identifying tooth abnormality parts, a training method, device, electronic device and computer-readable storage medium for a tooth abnormality part identification model. Background Art

[0002] In the digital oral scanning scenario, the doctor will compile and generate the corresponding oral examination report based on the patient's three-dimensional tooth model in order to analyze and summarize the patient's oral problems. This process usually requires the doctor to identify and judge oral diseases based on his or her own experience, and manually take screenshots and feedback to the patient. The operation is relatively cumbersome and requires a certain amount of time and manpower costs. In addition, the three-dimensional tooth model usually requires the doctor to manually rotate and observe it, and identify abnormal parts during this process. Improper selection of observation angles may lead to misdetection and missed detection of abnormal parts. Summary of the invention

[0003] In view of this, the present application provides a method for identifying tooth abnormality parts, a training method for a tooth abnormality part identification model, an electronic device, and a computer-readable storage medium.

[0004] Specifically, the present application is implemented through the following technical solutions:

[0005] According to a first aspect of an embodiment of the present application, a method for identifying an abnormal tooth part is provided, comprising:

[0006] Projecting the first tooth model to be identified according to a plurality of different projection viewing angles to obtain a plurality of first tooth images; wherein the plurality of different projection viewing angles are used to enable each tooth in the first tooth model to have projection results under at least two projection viewing angles in the plurality of first tooth images;

[0007] Using a pre-trained tooth abnormality part recognition model to perform tooth abnormality part recognition processing on a plurality of first tooth images, and obtain abnormal part recognition results corresponding to the plurality of first tooth images respectively; wherein the tooth abnormality part recognition model is obtained by supervised training based on a plurality of training samples, and the training samples include a plurality of second tooth images of a second tooth model at a plurality of different projection angles and an abnormal part label corresponding to each second tooth image;

[0008] The abnormal part recognition results respectively corresponding to the plurality of first tooth images are screened to determine the target abnormal part.

[0009] According to a second aspect of an embodiment of the present application, a method for training a tooth abnormality part recognition model is provided, comprising:

[0010] Acquire a plurality of training samples, the training samples comprising a plurality of second tooth images of the second tooth model at a plurality of different projection viewing angles and an abnormal part label corresponding to each second tooth image; in the plurality of second tooth images, each tooth in the second tooth model has projection results at at least two projection viewing angles; different training samples correspond to different second tooth models;

[0011] Inputting the training samples into the tooth abnormality recognition model to be trained, so that the tooth abnormality recognition model to be trained recognizes the tooth abnormality based on the multiple second tooth images, obtains the abnormality recognition results corresponding to the multiple second tooth images, and takes minimizing the difference between the abnormality recognition results corresponding to each second tooth image and the abnormality label corresponding to the second tooth image as the optimization goal, and trains the tooth abnormality recognition model to be trained;

[0012] The trained tooth abnormality part recognition model is applied to the tooth abnormality part recognition method of the first aspect. According to a third aspect of the embodiment of the present application, a tooth abnormality part recognition device is provided, comprising:

[0013] A first tooth image acquisition module, used for projecting the first tooth model to be identified according to a plurality of different projection viewing angles to obtain a plurality of first tooth images; wherein the plurality of different projection viewing angles are used to enable each tooth in the first tooth model to have projection results under at least two projection viewing angles in the plurality of first tooth images;

[0014] A tooth abnormality part recognition module is used to perform tooth abnormality part recognition processing on a plurality of first tooth images using a pre-trained tooth abnormality part recognition model to obtain abnormality part recognition results corresponding to the plurality of first tooth images respectively; wherein the tooth abnormality part recognition model is obtained by supervised training based on a plurality of training samples, and the training samples include a plurality of second tooth images of a second tooth model at a plurality of different projection angles and an abnormal part label corresponding to each second tooth image;

[0015] The target image determination module is used to screen and process the abnormal part recognition results corresponding to the plurality of first tooth images to determine the target abnormal part.

[0016] According to a fourth aspect of an embodiment of the present application, a training device for a tooth abnormality part recognition model is provided, comprising:

[0017] A training sample acquisition module, used to acquire a number of training samples, the training samples including a plurality of second tooth images of the second tooth model at a plurality of different projection viewing angles and an abnormal part label corresponding to each second tooth image; in the plurality of second tooth images, each tooth in the second tooth model has projection results at at least two projection viewing angles; different training samples correspond to different second tooth models;

[0018] A model training module is used to input training samples into a tooth abnormality part recognition model to be trained, so that the tooth abnormality part recognition model to be trained recognizes tooth abnormalities based on multiple second tooth images, obtains abnormal part recognition results corresponding to the multiple second tooth images, and trains the tooth abnormality part recognition model to be trained with the optimization goal of minimizing the difference between the abnormal part recognition results corresponding to each second tooth image and the abnormal part label corresponding to the second tooth image; wherein the trained tooth abnormality part recognition model is applied to the tooth abnormality part recognition device as mentioned in the third aspect.

[0019] According to a fifth aspect of an embodiment of the present application, there is provided an electronic device, including a memory, a processor, and executable instructions stored in the memory and executable on the processor;

[0020] When the processor executes the executable instructions, the steps in the method mentioned in the first aspect are implemented.

[0021] According to a sixth aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which computer instructions are stored. When the computer instructions are executed by a processor, the steps of any one of the methods described in the first aspect are implemented.

[0022] According to a seventh aspect of an embodiment of the present application, there is provided a computer program product, comprising a computer program / instruction, which, when executed by a processor, implements the steps of any one of the methods described in the first aspect.

[0023] The technical solution provided by the embodiments of the present application may have the following beneficial effects:

[0024] The training method of the tooth abnormal part recognition model provided in the embodiment of the present application constructs a rich training data set by acquiring multiple second tooth images of the second tooth model at different projection perspectives and labeling each second tooth image with a corresponding abnormal part label. The second tooth image at multiple projection perspectives can cover all parts of the teeth to ensure that no area is missed, thereby ensuring that after the model is supervised trained based on the training data set, the tooth abnormal part recognition model learns the characteristics of the teeth at different perspectives, can more comprehensively understand the morphology and structure of the teeth, and improves the adaptability of the model to tooth images at various projection perspectives in practical applications.

[0025] The method for identifying abnormal tooth parts provided in the embodiment of the present application can use the first tooth image under multiple different projection perspectives to perform AI (with the help of a pre-trained tooth abnormal part recognition model) prediction, thereby reducing the doctor's operating steps and improving recognition efficiency. Referring to the tooth images under multiple different projection perspectives can effectively avoid the problem of visual blind spots, improve the recognition accuracy of abnormal part recognition, and avoid or reduce the occurrence of abnormal parts being misdetected or missed. Finally, by screening and processing the abnormal part recognition results, it is ensured that the final target image carries valid lesion information.

[0026] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0028] Figure 1 It is a flowchart of a method for training a tooth abnormality part recognition model shown in an exemplary embodiment of the present application.

[0029] Figure 2 It is a schematic diagram of tooth position recognition shown in an exemplary embodiment of the present application.

[0030] Figure 3 It is a schematic diagram of a tooth model before and after alignment shown in an exemplary embodiment of the present application.

[0031] Figure 4 It is a schematic diagram showing a plurality of different projection viewing angles for a template tooth model according to an exemplary embodiment of the present application.

[0032] Figure 5 It is a schematic diagram showing the marking points after marking abnormal parts in the second tooth model according to an exemplary embodiment of the present application.

[0033] Figure 6 is a schematic diagram of an abnormal part label corresponding to a second tooth image shown in an exemplary embodiment of the present application.

[0034] Figure 7 It is a schematic diagram of a second tooth image and a second geometric information graph corresponding one to one under different projection viewing angles shown in an exemplary embodiment of the present application.

[0035] Figure 8 It is a rectangular frame indicating an abnormal part in each second tooth image and the confidence level of each rectangular frame shown in an exemplary embodiment of the present application.

[0036] Fig. 9 It is a flowchart of a method for identifying abnormal tooth parts shown in an exemplary embodiment of the present application.

[0037] Fig.10 It is a flowchart of another method for identifying abnormal tooth parts according to an exemplary embodiment of the present application.

[0038] Fig.11 It is a schematic diagram of a device for identifying abnormal parts of teeth shown in an exemplary embodiment of the present application.

[0039] Fig.12 It is a schematic diagram of a training device for a tooth abnormality part recognition model shown in an exemplary embodiment of the present application.

[0040] Fig.13 It is a structural schematic diagram of an electronic device shown in an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0041] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0042] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0043] It should be understood that although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0044] In digital oral scanning scenarios, for example, in the detection and identification process of common oral diseases such as wedge-shaped defects, gingival redness and swelling, and gingival atrophy, the doctor will generate a corresponding oral examination report based on the patient's 3D tooth model in order to analyze and summarize the patient's oral problems. This process usually requires the doctor to manually rotate the 3D tooth model, identify and judge oral diseases based on his or her own experience, and manually take screenshots and provide feedback to the patient. The operation is relatively cumbersome and requires a certain amount of time and manpower costs. In addition, improper selection of observation angles may lead to misdetection and missed detection of abnormal areas.

[0045] Here, the process of generating a 3D tooth model is described by way of example. An oral scanner can be used to obtain a 3D tooth model of a patient. A dentist operates the oral scanner and slowly moves the scanning head of the oral scanner in the patient's mouth to capture the image of the teeth. The scanning head emits a low-power light beam to illuminate the teeth. After the light beam is reflected back, it is captured by the scanner's sensor and the precise position of each point on the tooth surface is calculated through the triangulation principle. A 3D tooth model can be generated based on the data collected by the scanner. However, this is not limited to the above, and other methods can also be used to obtain a 3D tooth model.

[0046] Based on this, the embodiments of the present application provide a training method for a tooth abnormality part recognition model and a tooth abnormality part recognition method, which can assist doctors in quickly and accurately locating and identifying tooth abnormalities in an automated manner, and use tooth images under multiple different projection perspectives for AI (Artificial Intelligence) prediction, thereby reducing the doctor's operating steps and improving recognition efficiency. In addition, referring to tooth images under multiple different projection perspectives can effectively avoid visual blind spot problems, improve the recognition accuracy of abnormal part recognition, and avoid or reduce the occurrence of abnormal parts being misdetected or missed.

[0047] Next, the training process of the tooth abnormality recognition model is illustrated as follows:

[0048] The training method of the tooth abnormality identification model provided in the embodiment of the present application can be executed by electronic devices, including but not limited to physical servers, server machines, cloud servers, smart phones / mobile phones, tablet computers, personal digital assistants (PDAs), laptop computers, and desktop computers.

[0049] See also Figure 1 , a flow chart of a method for training a tooth abnormality part recognition model is shown, the method comprising:

[0050] In S101, a number of training samples are obtained, the training samples including a plurality of second dental images of the second dental model under a plurality of different projection perspectives and an abnormal part label corresponding to each second dental image; in the plurality of second dental images, each tooth in the second dental model has projection results under at least two projection perspectives; different training samples correspond to different second dental models.

[0051] In this step, a rich training data set is constructed by obtaining multiple second tooth images of the second tooth model at different projection angles and labeling each second tooth image with a corresponding abnormal part label. Such second tooth images at multiple projection angles can cover all parts of the teeth, ensuring that no area is missed, thereby ensuring that the model can learn the characteristics of the teeth at different angles and improving the model's adaptability to tooth images at various projection angles in practical applications.

[0052] And considering that some lesions may not be obvious at a certain angle, but very clear at another angle, different projection perspectives are designed so that each tooth in the second tooth model has projection results under at least two projection perspectives in multiple second tooth images. This allows comprehensive analysis of multi-angle images to avoid blind spots caused by a single perspective and improve the overall detection effect.

[0053] Each second tooth image is accompanied by a precise label of the abnormal part, which provides detailed truth values ​​to guide the training of the model, enabling the model to learn to identify dental lesions of different types and locations, improving the accuracy and reliability of recognition.

[0054] In S102, the training samples are input into the tooth abnormality part recognition model to be trained, and the tooth abnormality part recognition model to be trained performs tooth abnormality part recognition based on multiple second tooth images to obtain abnormal part recognition results corresponding to the multiple second tooth images respectively, and the tooth abnormality part recognition model to be trained is trained with the optimization goal of minimizing the difference between the abnormal part recognition results corresponding to each second tooth image and the abnormal part label corresponding to the second tooth image; wherein the trained tooth abnormality part recognition model is used to perform tooth abnormality part recognition processing on multiple first tooth images of the first tooth model under multiple different projection angles, thereby obtaining abnormal part recognition results corresponding to the multiple first tooth images respectively.

[0055] In this step, during the training process, through the second tooth image from multiple perspectives, the tooth abnormal part recognition model can more accurately locate the abnormal part, reduce false detection (mistaking the normal part for the lesion) and missed detection (failure to recognize the real lesion). The multi-perspective image provides more information, allowing the tooth abnormal part recognition model to make multiple judgments on the same part from different perspectives. In this way, even if there is an error in a certain perspective, it can be corrected through other perspectives. Through the above training process, the tooth abnormal part recognition model learns the different sides and angles of the teeth, and can more comprehensively understand the morphology and structure of the teeth, so that the tooth abnormal part recognition model can use the information from multiple perspectives for cross-validation when identifying lesions, thereby greatly improving the accuracy of recognition.

[0056] In this embodiment, the use of a trained tooth abnormality recognition model can greatly improve the efficiency and accuracy of dentists in diagnosing tooth lesions. The tooth abnormality recognition model can quickly process multiple tooth images of a three-dimensional tooth model at different projection angles and automatically identify possible lesion sites, thereby providing powerful auxiliary support for doctors.

[0057] In some embodiments, the process of acquiring multiple second tooth images of the second tooth model included in each training sample at different projection viewing angles is exemplarily described:

[0058] The electronic device performs tooth position recognition on each tooth in the second tooth model to determine the tooth position information of each tooth in the second tooth model; wherein the tooth position information corresponding to each tooth includes at least one of the tooth position number and tooth key point of each tooth. The tooth position number is a numbering system used to identify and name teeth, so that the position of each tooth can be quickly and accurately determined, such as maxillary, mandibular, left and right sides, front and back positions, etc. Tooth key points refer to specific marks or marking points on teeth, which are usually used to describe the structure, anatomical features and positioning of teeth during treatment. For example, tooth key points are used to indicate important structural features, such as the demarcation line, depression or protrusion of the crown and root.

[0059] See for example Figure 2 , showing the tooth position number corresponding to each tooth obtained by tooth position recognition of a three-dimensional tooth model ( Figure 2 ) and tooth key points (tooth key points are Figure 2 ).

[0060] Next, the electronic device aligns the second tooth model with the template tooth model based on the tooth position information of each tooth in the second tooth model and the tooth position information of each tooth in the predetermined template tooth model to obtain the aligned second tooth model. Exemplarily, the ICP (Iterative Closest Point) algorithm can be used to achieve the alignment between the two. First, at least some of the key points in the second tooth model are selected, and the key points closest to them are found in the template tooth model. The optimal rigid body transformation between the second tooth model and the template tooth model is calculated based on each pair of matching key points, including rotation and translation, such as by minimizing the distance between each pair of matching key points to calculate the optimal transformation matrix; the estimated transformation matrix is ​​applied to the second tooth model so that the second tooth model is transformed relative to the template tooth model to align with the template tooth model as much as possible. Multiple iterations can be performed, and in each iteration, the key points are reselected and the transformation matrix is ​​recalculated in the above manner, and the transformation matrix is ​​applied to the second tooth model to gradually optimize the position and rotation applied to the second tooth model so that it is better aligned with the template tooth model. During the alignment process, special attention is paid to the matching of key points of the teeth to ensure accurate correspondence between tooth position numbers and key points.

[0061] See for example Figure 3 , showing a schematic diagram of a three-dimensional tooth model before and after alignment with a template tooth model.

[0062] After obtaining the aligned second dental model, the aligned second dental model can be projected from three-dimensional space to a two-dimensional plane according to a plurality of different projection viewing angles preset corresponding to the template dental model to obtain a plurality of second dental images; wherein the plurality of different projection viewing angles preset corresponding to the template dental model are used to enable each tooth in the second dental model to have projection results under at least two projection viewing angles in the plurality of second dental images; and one projection viewing angle corresponds to one second dental image.

[0063] See for example Figure 4 , Figure 4 Shows multiple different projection angles for the template tooth model, based on Figure 4 The multiple different projection perspectives shown project the aligned second tooth model from three-dimensional space to two-dimensional space, so that each tooth in the second tooth model has at least projection results under three projection perspectives of front, back and top in multiple second tooth images, thereby effectively avoiding the problem of visual blind spots.

[0064] In this embodiment, model alignment is performed based on the tooth position information corresponding to each tooth to ensure that the second tooth model is highly consistent with the template tooth model in space, and then the aligned second tooth model is projected according to a plurality of different projection perspectives corresponding to the template tooth model to ensure that each projection follows the same standard. Through standardized perspectives, consistency of all generated tooth images can be guaranteed to facilitate subsequent analysis and diagnosis, thereby improving overall reliability and accuracy.

[0065] In some embodiments, the following is an exemplary description of the process of obtaining the abnormal part label corresponding to each second tooth image in the training sample: the electronic device can display the second tooth model corresponding to each training sample in the display interface, so that the user can quickly view and select. The user can determine whether there is a lesion area in the second tooth model based on personal experience and mark it. The electronic device can respond to the marking operation on the second tooth model and add a marking point at the marking position indicated by the marking operation to obtain the second tooth model with the marking point, which can significantly improve the marking efficiency of the abnormal part and ensure the accurate positioning of the abnormal part.

[0066] Exemplarily, the user can select the abnormal part in the second tooth model for annotation by means of a mouse, keyboard, stylus or skin touch, and the position of the abnormal part is the annotation position. For example, the user can click or long press the annotation position to perform annotation operations, so that the electronic device can respond to the annotation operation, add annotation points at the annotation position and display them, for example Figure 5 The red marking point shown. The marking method provided in this embodiment is simple and direct, which is conducive to simplifying the marking steps of the user and improving the marking efficiency.

[0067] After the second tooth model is labeled, the electronic device can project the second tooth model carrying the marked points from the three-dimensional space to the two-dimensional plane according to the above-mentioned different projection angles to obtain multiple second tooth images carrying the marked points. The projection process is similar to the above-mentioned projection process for obtaining multiple second tooth images, which will not be repeated here; then the electronic device can generate a graphic frame of a preset size for indicating the abnormal part with the marked points in each second tooth image carrying the marked points as the center, and determine the graphic frame as the abnormal part label corresponding to the second tooth image; wherein, the graphic frame for indicating the abnormal part can be a rectangular frame, a circular frame, a polygonal frame or a graphic frame of any other shape, and this embodiment does not impose any restrictions on this, for example Figure 6The green rectangular box shown is the abnormal part label corresponding to the second tooth image. In this embodiment, a graphic box of a preset size is generated with the marking point as the center to indicate the abnormal part, so as to automatically generate the abnormal part label corresponding to the image and reduce the workload of manual labeling; and the automatically generated lesion labels can be standardized to ensure consistency and repeatability, thereby improving the quality and utilization value of the data.

[0068] In some embodiments, the abnormal part label corresponding to each second tooth image also includes an abnormal type corresponding to each graphic box; the abnormal type includes at least one of a wedge-shaped defect, tartar, pigmentation, red and swollen gums, caries, and gum atrophy. In the above-mentioned process of marking the abnormal part, the electronic device can also obtain the abnormal type corresponding to each marked point (or the graphic box generated by each marked point) in response to the marking operation on the second tooth model. Exemplarily, the user can input the abnormal type information corresponding to each marked point by means of a mouse, keyboard, stylus, or skin touch.

[0069] In some embodiments, in order to further improve the prediction accuracy, in addition to referring to multiple second tooth images of the second tooth model at different projection angles, the geometric information of the three-dimensional points in the second tooth model may be further referred to. In addition to including multiple second tooth images of the second tooth model at different projection angles and abnormal part labels corresponding to each second tooth image, each training sample also includes multiple second geometric information graphs of the geometric information of the three-dimensional points in the second tooth model at the different projection angles; wherein the multiple second geometric information graphs correspond one-to-one to the multiple second tooth images.

[0070] The three-dimensional points in the second tooth model include at least two types of geometric information; the electronic device may obtain a plurality of second geometric information images in the following manner:

[0071] For each type of geometric information of the three-dimensional point, the electronic device can rescale the geometric information of this type to within a preset numerical range, and project the rescaled geometric information onto a two-dimensional plane according to the above-mentioned different projection perspectives to obtain multiple first images; in the process of projecting the rescaled geometric information onto a two-dimensional plane, since the three-dimensionality of the second tooth model may not directly correspond to the pixels of the first image, interpolation is required, and the purpose of interpolation is to ensure that each pixel on the first image can obtain a reasonable geometric information value. Then, the electronic device can map the multiple first images to one of the color channels of the specified color mode to obtain multiple second images; wherein different types of geometric information of the three-dimensional point correspond to different color channels; the preset numerical range is the color representation range of the specified color mode. Finally, after obtaining multiple second images corresponding to all types of geometric information of the three-dimensional point, the electronic device fuses the second images belonging to the same projection perspective to obtain multiple second geometric information maps.

[0072] The specified color mode includes but is not limited to RGB (red, green, blue) mode, CMYK (cyan, magenta, yellow, black) mode, and HSV / HSB (hue, saturation, brightness) mode, etc., and can be selected based on the actual application scenario.

[0073] For example, the color representation range of each color channel in the RGB mode is 0~255, where 0 represents the lowest intensity of the color channel (i.e., off or no light), and 255 represents the highest intensity of the color channel (i.e., maximum brightness). For example, pure red can be represented as (255, 0, 0), pure green as (0, 255, 0), and pure blue as (0, 0, 255). White is when all channels reach maximum brightness, i.e., (255, 255, 255), and black is when all channels are off, i.e., (0, 0, 0).

[0074] For example, the CMYK (Cyan, Magenta, Yellow, Black) model is a color model used for printing that uses four basic colors to mix and produce a wide range of colors. Each color is usually represented from 0% to 100%, representing the concentration or intensity of the color: (1) Cyan: 0% to 100%; (2) Magenta: 0% to 100%; (3) Yellow: 0% to 100%; (4) Key / Black: 0% to 100%. In the CMYK model, 0% represents the lowest intensity of the color (or the absence of the color), while 100% represents the highest intensity of the color. For example, pure cyan can be represented as (100%, 0%, 0%, 0%), while pure black can be represented as (0%, 0%, 0%, 100%).

[0075] Exemplarily, the geometric information of the 3D point includes at least one of the following types: the curvature, depth and normal vector of the 3D point. The curvature of the 3D point is usually the average of the maximum and minimum curvatures of the 3D point, the depth of the 3D point is the depth relative to a reference object (such as a camera), and the normal vector of the 3D point is the normal vector relative to the gum base.

[0076] The curvature, depth, and normal vector of a 3D point can be mapped to different color channels of the RGB model respectively.

[0077] For example, after acquiring the curvature, depth and normal vector of the three-dimensional point in the second tooth model, the electronic device can rescale the curvature, depth and normal vector of the three-dimensional point in the second tooth model to the range of 0 to 255, so as to be represented on the two-dimensional image. Then, the rescaled curvature, depth and normal vector of the three-dimensional point are projected onto the two-dimensional plane according to the above-mentioned different projection angles, and a plurality of first images corresponding to the curvature of the three-dimensional point, a plurality of first images corresponding to the depth of the three-dimensional point, and a plurality of first images corresponding to the normal vector of the three-dimensional point are obtained.

[0078] The electronic device maps the multiple first images corresponding to the curvature of the three-dimensional point to the red channel of the RGB mode to obtain multiple second images corresponding to the curvature of the three-dimensional point; maps the multiple first images corresponding to the depth of the three-dimensional point to the green channel of the RGB mode to obtain multiple second images corresponding to the depth of the three-dimensional point; and maps the multiple first images corresponding to the normal vector of the three-dimensional point to the blue channel of the RGB mode to obtain multiple second images corresponding to the normal vector of the three-dimensional point. It can be understood that all the second images are single-channel images.

[0079] After obtaining multiple second images corresponding to the curvature, depth, and normal vector of the three-dimensional point, the second images belonging to the same projection perspective are fused to obtain multiple first geometric information maps. For example, there are three projection perspectives, and the multiple second images corresponding to the curvature of the three-dimensional point include {a1, a2, a3}, the multiple second images corresponding to the depth of the three-dimensional point include {b1, b2, b3}, and the multiple second images corresponding to the normal vector of the three-dimensional point include {c1, c2, c3}. Assuming that a1, b1, and c1 belong to the same projection perspective, a1, b1, and c1 are fused into a second geometric information map; similarly, a2, b2, and c2 belong to the same projection perspective, a2, b2, and c2 are fused into a second geometric information map; a3, b3, and c3 belong to the same projection perspective, a3, b3, and c3 are fused into a second geometric information map, and there are a total of 3 second geometric information maps.

[0080] The second tooth model has a one-to-one correspondence with the second tooth images and the second geometric information graphs. Figure 7 , showing the second tooth images and the second geometric information graph corresponding to each other under different projection viewing angles.

[0081] For each training sample, the electronic device can superimpose each second tooth image in the training sample and the second geometric information map corresponding to the second tooth image into a target image, and the target image is represented by at least four channels, of which three channels are used to describe the second tooth image, and the remaining at least one channel is used to describe the second geometric information map corresponding to the second tooth image. The number of channels of the target image is positively correlated with the type of geometric information of the three-dimensional point. The more types of geometric information, the more channels of the target image.

[0082] In one example, the geometric information of the three-dimensional point includes the following three types: curvature, depth and normal vector of the three-dimensional point. Then the target image is represented by six channels, three of which are used to describe the second tooth image, and the remaining three channels are used to describe the second geometric information map corresponding to the second tooth image.

[0083] During the model training process, the electronic device inputs the training samples into the tooth abnormality recognition model to be trained, so that the tooth abnormality recognition model to be trained recognizes the tooth abnormality based on multiple target images, and obtains the abnormality recognition results corresponding to the multiple target images respectively. The tooth abnormality recognition model to be trained is trained with the optimization goal of minimizing the difference between the abnormality recognition results corresponding to each target image and the abnormality label corresponding to the target image.

[0084] This embodiment takes into account the geometric information of the three-dimensional points in the tooth model during the model training process. The target image obtained by superimposing the second tooth image and the second geometric information map significantly improves the prediction accuracy, robustness, and versatility of the model, which not only improves the overall performance of the model, but also provides strong support for clinical applications. Geometric information can provide additional spatial structure and shape features, which help to more accurately locate and identify the lesion area. For example, geometric information can help the model perform more refined feature extraction, including subtle changes and shape features on the tooth surface, which is particularly important for lesion identification. The addition of geometric information enables the model to not only identify surface lesions, but also make judgments on potential structural changes.

[0085] Exemplarily, the electronic device can input training samples into a tooth abnormality part recognition model to be trained, and execute the following process in a loop until the optimization target is met: using the current tooth abnormality part recognition model to recognize tooth abnormalities based on multiple target images, and obtaining abnormal part recognition results corresponding to the multiple target images; judging whether the optimization target is met based on the difference between the abnormal part recognition results corresponding to each target image and the abnormal part label corresponding to the target image; if the optimization target is not met, adjusting the model parameters of the current tooth abnormality part recognition model based on the difference between the abnormal part recognition results corresponding to each target image and the abnormal part label corresponding to the target image, and using the adjusted tooth abnormality part recognition model as the current tooth abnormality part recognition model in the next loop.

[0086] Among them, the abnormal part recognition results corresponding to each target image include at least one predicted graphic box for indicating the abnormal part and the confidence of each predicted graphic box; confidence refers to the degree of confidence or accuracy of the machine learning model in its prediction results. In machine learning and deep learning, the model can not only give a prediction result, but also be accompanied by a confidence value to indicate the degree of certainty of the model in the prediction. Confidence is usually a probability value or score, which reflects the degree of trust the model has in a certain prediction result (that is, the location of the graphic box). High confidence means that the model believes that its prediction result is accurate, and low confidence means that the model is not very sure about its prediction result, and there may be a certain degree of uncertainty.

[0087] See for example Figure 8 , showing the rectangular boxes used to indicate abnormal parts in each second tooth image and the confidence of each rectangular box. The green marking point in the middle of the rectangular box is the marking point used to indicate the lesion area determined in the above labeling process.

[0088] Exemplarily, the abnormal part identification results corresponding to each target image also include the predicted abnormality type corresponding to each graphic box; wherein the predicted abnormality type includes at least one of wedge-shaped defects, tartar, pigmentation, caries, gingival redness and swelling, and gingival atrophy.

[0089] From the above description, it can be seen that the abnormal part label corresponding to each target image is a graphic box used to indicate the abnormal part (for the convenience of distinction, hereinafter referred to as the graphic box label) and a pre-labeled abnormal type (for the convenience of distinction, hereinafter referred to as the abnormal type label). Then the intersection-and-union ratio between the predicted graphic box corresponding to each target image and the graphic box label corresponding to the target image can be calculated, and the difference between the predicted abnormal type and the abnormal type label can be compared; the larger the intersection-and-union ratio, the higher the degree of overlap between the two, and if the predicted abnormal type and the abnormal type label are the same, the smaller the difference between the two, and the closer to the optimization target; the smaller the intersection-and-union ratio, the lower the degree of overlap between the two, and if the predicted abnormal type and the abnormal type label are different, the greater the difference between the two, and the need to readjust the model parameters to continue training.

[0090] It is understandable that this embodiment does not impose any restrictions on the structure of the tooth abnormal part recognition model and the loss function used in the training process, and can be specifically set according to the actual application scenario. For example, the structure of the tooth abnormal part recognition model includes but is not limited to a convolutional neural network structure, a U-Net neural network, a residual network, or a model based on a self-attention mechanism (Transformer), etc. For example, the loss function includes but is not limited to a cross entropy loss function, a Dice loss function, a mean square error loss function, an IoU loss function (Intersection over Union Loss), or a bounding box regression loss, etc.

[0091] It is understandable that the electronic device used to train the tooth abnormality part recognition model and the electronic device used to perform the tooth abnormality part recognition method can be the same device or different devices, and this embodiment does not impose any restrictions on this. For example, the tooth abnormality part recognition model can be trained by a certain electronic device, and then the trained tooth abnormality part recognition model can be deployed in another electronic device used to perform the tooth abnormality part recognition method.

[0092] The following is an exemplary description of the method for identifying abnormal tooth parts provided in the embodiment of the present application:

[0093] See also Fig. 9 , a schematic flow chart of a method for identifying abnormal tooth parts is shown, which can be performed by an electronic device, including but not limited to a physical server, a server machine, a cloud server, a smart phone / mobile phone, a tablet computer, a personal digital assistant (PDA), a laptop computer, and a desktop computer. The method includes:

[0094] In S901, the first tooth model to be identified is projected according to a plurality of different projection viewing angles to obtain a plurality of first tooth images; wherein the plurality of different projection viewing angles are used to ensure that each tooth in the first tooth model has projection results under at least two projection viewing angles in the plurality of first tooth images.

[0095] In this step, multiple two-dimensional tooth images (i.e., multiple first tooth images) can be generated by projecting the first tooth model at different projection angles. These first tooth images capture the tooth structure and features at different angles, which are helpful for comprehensive analysis and understanding of the tooth morphology, avoiding blind spots caused by a single angle, and improving the overall detection effect. Each projection angle corresponds to a first tooth image.

[0096] In S902, a pre-trained tooth abnormality part recognition model is used to perform tooth abnormality part recognition processing on multiple first tooth images to obtain abnormal part recognition results corresponding to the multiple first tooth images respectively; wherein the tooth abnormality part recognition model is obtained through supervised training based on a number of training samples, and the training samples include multiple second tooth images of the second tooth model under multiple different projection perspectives and abnormal part labels corresponding to each second tooth image.

[0097] In this step, a pre-trained tooth abnormality identification model is used to identify abnormal parts of multiple first tooth images. Referring to the first tooth images under multiple different projection perspectives can effectively avoid the problem of visual blind spots and can quickly and accurately locate and identify possible dental lesions.

[0098] In S903, the abnormal part recognition results corresponding to the plurality of first tooth images are screened to determine the target abnormal part.

[0099] In this step, the target image carrying the target abnormal part information is selected by screening the abnormal part recognition results. This screening can effectively reduce misidentification and erroneous results, ensure that the final target image carries valid lesion information, and help further clinical decision-making and treatment planning.

[0100] The method for identifying abnormal tooth parts provided in this embodiment combines multi-angle projection technology and a deep learning tooth lesion recognition model, effectively improving the efficiency and accuracy of dental health management. Multiple first tooth images generated by multi-angle projection, combined with a high-precision tooth lesion recognition model, can accurately detect tooth lesion information, avoid or reduce the occurrence of abnormal parts being misdetected or missed, and provide dental professionals with more powerful treatment basis and decision support.

[0101] In some embodiments, with respect to S901, similar to the training process, the electronic device can perform tooth position recognition on each tooth in the first tooth model, and determine the tooth position information of each tooth in the first tooth model; the tooth position information corresponding to each tooth includes at least one of the tooth position number and the tooth key point of each tooth. Then, based on the tooth position information of each tooth in the first tooth model and the tooth position information of each tooth in the predetermined template tooth model, the first tooth model is aligned with the template tooth model to obtain the aligned first tooth model; finally, according to the preset multiple different projection angles corresponding to the template tooth model, the aligned first tooth model is projected from the three-dimensional space to the two-dimensional plane to obtain multiple first tooth images.

[0102] In this embodiment, by aligning the first tooth model with the template tooth model, it can be ensured that the tooth data obtained between different individuals or at different time points of the same patient have consistent spatial positioning. The alignment process can standardize the tooth models from different data sources, making subsequent image processing and analysis more reliable and repeatable. Based on multiple preset different projection angles, the aligned first tooth model is projected from three-dimensional space to a two-dimensional plane to generate multiple tooth images. These images cover different angles and views of the teeth, providing comprehensive tooth structure information.

[0103] In some embodiments, the geometric information of the three-dimensional points in the tooth model can also be integrated to predict the abnormal part of the tooth, thereby improving the prediction accuracy. Fig.10 The tooth abnormality part recognition method provided in the embodiment of the present application further includes S904, obtaining geometric information of the three-dimensional point in the first tooth model; encoding and projecting the geometric information of the three-dimensional point according to different projection angles to obtain multiple first geometric information maps; the multiple first geometric information maps correspond to the multiple first tooth images one by one. The geometric information of the three-dimensional point includes at least one of the following types: curvature, depth, and normal vector of the three-dimensional point.

[0104] Exemplarily, the three-dimensional points in the first tooth model include at least two types of geometric information. In the process of acquiring multiple first geometric information maps, similar to the training process, for each type of geometric information of the three-dimensional point, the electronic device can rescale the geometric information of this type to a preset numerical range, and project the rescaled geometric information to a two-dimensional plane according to different projection perspectives to obtain multiple first images; and, respectively map the multiple first images to one of the color channels of the specified color mode to obtain multiple second images; wherein different types of geometric information of the three-dimensional point correspond to different color channels; the preset numerical range is the color representation range of the specified color mode; finally, after obtaining multiple second images corresponding to all types of geometric information of the three-dimensional point, the second images belonging to the same projection perspective are fused to obtain multiple first geometric information maps.

[0105] Exemplarily, the geometric information of the three-dimensional point includes at least one of the following: curvature, depth, and normal vector of the three-dimensional point.

[0106] In this embodiment, the electronic device scales the geometric information of each type of three-dimensional point to a preset numerical range, ensuring that all geometric information data is on a uniform scale, which is conducive to subsequent data processing and analysis. In addition, the geometric information of each type of three-dimensional point corresponds to a different color channel. This color coding method helps to separate different types of geometric information so that effective data analysis and identification can be performed in subsequent steps. Finally, multiple second images of the same projection perspective are fused to obtain multiple first geometric information maps. These first geometric information maps contain different types of geometric information of the same tooth part, and each type of geometric information is encoded with a different color channel. This comprehensive image can provide a comprehensive data perspective to help the tooth abnormality part recognition model accurately locate and analyze potential problems and lesions on the teeth.

[0107] See also Fig.10 , step S902 includes S9021 and S9022.

[0108] In S9021, each first tooth image and a first geometric information map corresponding to the first tooth image are superimposed to form a target image.

[0109] The target image is represented by at least four channels, three of which are used to describe the first tooth image, and the remaining at least one channel is used to describe the first geometric information map. The number of channels of the target image is positively correlated with the type of geometric information of the three-dimensional point. The more types of geometric information, the more channels of the target image.

[0110] In one example, the geometric information of the three-dimensional point includes the following three types: curvature, depth and normal vector of the three-dimensional point. Then the target image is represented by six channels, three of which are used to describe the first tooth image, and the remaining three channels are used to describe the first geometric information map corresponding to the first tooth image.

[0111] In S9022, the plurality of target images are input into the dental abnormality recognition model, so that the dental abnormality recognition model performs dental abnormality recognition processing on the plurality of target images to obtain abnormality recognition results corresponding to the plurality of target images respectively.

[0112] This embodiment inputs multiple target images into the tooth abnormality part recognition model, comprehensively considering the morphology, structure and performance of the teeth at different viewing angles. This comprehensive analysis process helps to improve the prediction accuracy.

[0113] In some embodiments, the electronic device inputs multiple target images into a tooth abnormal part recognition model, so that the tooth abnormal part recognition model extracts color texture features and geometric features from each target image, determines each part in the oral environment by analyzing the geometric features, and classifies based on the color texture features and geometric features corresponding to each part in the oral environment, to obtain abnormal part recognition results corresponding to each target image; wherein each part in the oral environment includes at least one of the occlusal surface of the tooth, the non-occlusal surface of the tooth, the junction between the tooth and the gum, the gum, the labial side, and the lingual side. In this embodiment, the model recognizes each part in the oral environment, can comprehensively analyze the health status of the teeth from multiple angles, and can more accurately identify abnormal parts in the teeth and the oral cavity by combining color texture features and geometric features, thereby improving the accuracy of diagnosis.

[0114] Exemplarily, the geometric features extracted from each target image by the tooth abnormality recognition model include at least one of depth features, curvature features, and normal vector features.

[0115] Depth features can help distinguish the occlusal and non-occlusal surfaces of teeth. The occlusal surface is the highest point of the tooth and the area in contact with the opposing teeth, while the non-occlusal surface is the side of the tooth or the area not directly involved in the occlusion; the occlusal surface usually has a lower depth value, while the non-occlusal surface is relatively high. Depth features can also be used to identify the junction between teeth and gums. The gums usually surround the base of the teeth and have a higher depth value, while the surface depth of the teeth themselves is lower. By analyzing the changes in the depth gradient, the boundary between the teeth and gums can be determined.

[0116] The occlusal surface usually shows a high curvature change, especially in the cusp and pit and fissure areas, which have high curvature values ​​due to the high occlusal pressure and complex morphology. The non-occlusal surface of the tooth has a small curvature change and is relatively smooth, especially on the labial and lingual sides of the tooth. These relatively smooth areas can be distinguished by analyzing the curvature values. The gingival margin usually shows an area with a sharp change in curvature, which reflects the transition from the tooth surface to the gingival surface.

[0117] The normal vector on the occlusal surface usually points in the occlusal direction of the tooth, that is, perpendicular to the occlusal surface of the tooth; on the non-occlusal surface, the normal vector is tilted or points in other directions. On the labial and lingual sides, the normal vector generally points to the outside or inside of the tooth. These features are significantly different from the normal vector on the occlusal surface and help to accurately distinguish these areas. At the junction of the tooth and the gum, the normal vector may undergo a significant change in direction. This change is caused by the different geometric shapes when the tooth surface transitions to the gum surface.

[0118] Therefore, various parts in the oral environment can be determined by analyzing at least one of the depth features, curvature features, and normal vector features. Furthermore, the above three geometric features can be combined to effectively identify and distinguish various parts in the oral environment. The depth feature provides basic information on height and position, the curvature feature reveals the complexity of the surface morphology, and the normal vector feature provides detailed information on the surface orientation. Combining these geometric features, the occlusal surface, non-occlusal surface, the junction between the teeth and gums, the gums, the labial side and the lingual side can be accurately determined, thereby providing a basis for further abnormality detection and diagnosis.

[0119] For example, after determining various parts in the oral environment, the dental abnormality part recognition model may classify various parts in the oral environment based on:

[0120] (1) If the part is the occlusal surface of the tooth, the color texture characteristics corresponding to the part indicate that it belongs to the dark range, and the geometric characteristics corresponding to the part show a changing trend, it is determined that there is a wedge-shaped defect in the part.

[0121] Specifically, wedge-shaped defects usually manifest as a decrease in depth in certain areas of the tooth surface, that is, the surface is eroded or missing, resulting in a low local depth value. Wedge-shaped defects are usually accompanied by a sharp change in curvature, especially in the damaged edge area, where the curvature value increases significantly. The formation of a wedge-shaped defect usually causes a change in the direction of the normal vector, especially at the edge of the defect, where the normal vector may no longer point to the normal occlusal direction of the tooth. Due to surface damage or exposure of inner tissues such as dentin, the wedge-shaped defect area is usually dark. Therefore, when the occlusal surface is dark and the geometric features (such as reduced depth, increased curvature, and normal vector change) are abnormal, the model can determine that a wedge-shaped defect exists in this area.

[0122] (2) If the location is the junction of the teeth and gums, the color texture characteristics corresponding to the location indicate that it belongs to the light color range, and the geometric characteristics corresponding to the location show a changing trend, it is determined that dental calculus exists at the location.

[0123] Specifically, tartar buildup at the tooth-gingival interface will cause the depth value in that area to increase because the tartar forms an additional covering layer. Tartar often causes changes in the surface curvature at the tooth-gingival interface, especially in areas with more buildup, where the curvature value increases. Because tartar covers the tooth surface, the normal vector may slightly deviate from the direction of the normal tooth surface. Tartar is often light or white, especially when newly formed. Therefore, when the color of the tooth-gingival interface is light and the geometric features (such as increased depth, increased curvature, and slightly deviated normal vector) are abnormal, the model can determine that tartar is present in that area.

[0124] (3) If the site is the non-occlusal surface, labial side or lingual side of the tooth, the color and texture characteristics corresponding to the site indicate that it belongs to the dark range and the geometric characteristics corresponding to the site show a stable trend, it is determined that pigmentation exists in the site.

[0125] Specifically, pigmentation does not usually cause significant changes in the depth of the tooth surface, so the depth feature remains stable. Pigmentation also does not significantly change the morphology of the tooth surface, so the curvature feature should also remain stable. Pigmentation has little effect on the geometric morphology of the tooth surface, and the direction of the normal vector generally does not change. Pigmentation usually manifests as a darker color on the tooth surface, which is common on the labial or lingual side. Therefore, when the color of the non-occlusal surface, labial or lingual area is dark but the geometric features (such as depth, curvature, normal vector) remain stable, the model can determine that pigmentation exists in that area.

[0126] (4) If the site is the non-occlusal surface, labial side or lingual side of the tooth, the color texture characteristics corresponding to the site indicate that it belongs to the dark range and the geometric characteristics corresponding to the site show a changing trend, it is determined that caries exists in the site.

[0127] Specifically, caries usually leads to a decrease in the depth of the tooth surface, that is, the tooth surface is corroded and pits are formed. Caries areas are usually accompanied by obvious changes in curvature, especially at the edge of the caries, where the curvature value may increase. Due to the pits formed by caries, the direction of the normal vector in this area will change, pointing to the inside of the pit. Caries areas are usually dark, which is caused by the loss of material on the tooth surface and the exposure of diseased tissue. Therefore, when the color of the non-occlusal surface, labial or lingual area is dark and the geometric features (such as reduced depth, increased curvature, and normal vector change) are abnormal, the model can determine that caries exists in this area.

[0128] (5) If the site is the gums, the color texture feature corresponding to the site indicates that it belongs to the red range, and the geometric feature corresponding to the site shows a changing trend, it is determined that the site has red and swollen gums or gum atrophy.

[0129] Specifically, gingival redness usually results in increased depth values ​​for the gingival area, while gingival recession results in decreased depth values. Gum redness may result in increased local curvature, while gingival recession results in decreased or smoothed curvature. The normal vector of the gingival area may slightly deviate from the normal direction due to redness or recession. Gum redness areas are usually red, while recession gums may become pale or lighter in color. Therefore, when the color of the gingival area is red and the geometric features (such as increased or decreased depth, curvature changes, normal vector deviation) are abnormal, the model can determine that gingival redness or recession exists in that area.

[0130] By jointly analyzing these geometric features and color and texture features, the tooth abnormality recognition model can effectively identify and classify abnormalities in different parts of the oral cavity, thereby improving the accuracy and reliability of diagnosis.

[0131] In some embodiments, the abnormal part recognition result corresponding to each first tooth image (or target image) includes at least one graphic frame for indicating the abnormal part superimposed on the first tooth image (or target image). The following is an exemplary description using the first tooth image:

[0132] After obtaining the abnormal part recognition results corresponding to the plurality of first tooth images, the electronic device performs back-projection on the abnormal part recognition results of each of the plurality of first tooth images, so as to present a graphic frame indicating the abnormal part in the first tooth model; back-projection refers to the process of mapping the recognition results in the two-dimensional image back to the three-dimensional model, and the abnormal part recognition results of each first tooth image will be "back-projected" into the three-dimensional tooth model to accurately locate the position of the abnormal part in the three-dimensional space. Then, the electronic device counts the number of graphic frames indicating the same abnormal part presented in the first tooth model; if the number of graphic frames indicating the same abnormal part presented in the first tooth model exceeds the preset number, the abnormal part is determined to be the target abnormal part. In this embodiment, by back-projecting the recognition results of the plurality of first tooth images corresponding to the plurality of different projection angles, the information of different angles can be integrated, thereby reducing the recognition error caused by a single angle, and only when the first tooth images at different angles recognize the same abnormal part, it will be determined as the target abnormal part, which greatly improves the accuracy of abnormal recognition.

[0133] Among them, the preset number is greater than half of the total number of multiple different projection viewing angles. That is to say, when the same abnormal part is identified in the first tooth images corresponding to more than half of the projection viewing angles, it will be confirmed as the target abnormal part. This can effectively filter out misjudgments under a single viewing angle caused by noise or image quality problems, thereby improving the reliability of the overall recognition results.

[0134] Exemplarily, the abnormal part recognition results corresponding to each first tooth image also include the abnormal type corresponding to each graphic frame; wherein the abnormal type includes at least one of wedge-shaped defects, dental calculus, pigmentation, caries, gingival redness and swelling, and gingival atrophy. In the process of determining the target abnormal part, if the number of graphic frames used to indicate the same abnormal part presented in the first tooth model exceeds a preset number, and the abnormal types corresponding to all graphic frames used to indicate the same abnormal part are the same abnormal type, the abnormal part is determined to be the target abnormal part. In this embodiment, not only the number of graphic frames projected at multiple angles is considered, but also the abnormal types indicated by these graphic frames are required to be consistent. This can prevent confusion of different abnormal types due to recognition errors or noise, ensure that the target abnormal part finally determined is verified multiple times and the abnormal type is clear, and improve the accuracy of abnormal recognition.

[0135] In some embodiments, after determining the target abnormal part, the electronic device can determine the non-occluded viewing angle of the target abnormal part on the first tooth model; wherein the non-occluded viewing angle is a viewing angle at which the target abnormal part can be observed on the first tooth model. Specifically, the non-occluded viewing angle refers to a viewing angle at which the target abnormal part is not obstructed by other teeth or oral structures and can be clearly seen when observed from a certain angle of the three-dimensional tooth model. This viewing angle can ensure that during the observation and diagnosis process, the details of the target abnormal part can be fully displayed, avoiding information loss or misjudgment due to viewing angle problems.

[0136] When the first tooth model is displayed on the display interface, the non-occluded perspective is used as the screenshot perspective of the display interface, and the display interface is screenshotted to obtain a two-dimensional screenshot containing the target abnormal part. After determining the screenshot perspective, the electronic device screenshots the display interface to generate a two-dimensional screenshot containing the target abnormal part. The screenshot not only retains the realism of the three-dimensional model, but can also be easily shared or used for further analysis. For example, the two-dimensional screenshot can be embedded in a report, used for patient communication, or saved as part of a medical record. Because the screenshot perspective is a carefully selected non-occluded perspective, these screenshots can clearly show the target abnormal part in any situation, making it easier for doctors and patients to understand the problem.

[0137] It is not difficult to understand that the schemes described in the above embodiments can be combined when there is no conflict, and they are not listed one by one in the embodiments of the present application.

[0138] In some embodiments, see Fig.11 The present application also provides a device for identifying abnormal tooth parts, including:

[0139] The first tooth image acquisition module 1101 is used to project the first tooth model to be identified according to multiple different projection perspectives to obtain multiple first tooth images; wherein the different projection perspectives are used to make each tooth in the first tooth model have projection results under at least two projection perspectives in the multiple first tooth images.

[0140] The tooth abnormality part recognition module 1102 is used to use a pre-trained tooth abnormality part recognition model to perform tooth abnormality part recognition processing on multiple first tooth images to obtain abnormal part recognition results corresponding to the multiple first tooth images respectively; wherein the tooth abnormality part recognition model is obtained through supervised training based on a number of training samples, and the training samples include multiple second tooth images of the second tooth model under multiple different projection perspectives and abnormal part labels corresponding to each second tooth image.

[0141] The target abnormal part determination module 1103 is used to screen and process the abnormal part recognition results corresponding to the plurality of first tooth images to determine the target abnormal part.

[0142] In one implementation, the first dental image acquisition module 1101 includes: performing tooth position identification on each tooth in the first dental model to determine the tooth position information of each tooth in the first dental model; aligning the first dental model with the template dental model based on the tooth position information of each tooth in the first dental model and the tooth position information of each tooth in the predetermined template dental model to obtain an aligned first dental model; projecting the aligned first dental model according to a plurality of different projection viewing angles corresponding to the preset template dental model to obtain a plurality of first dental images.

[0143] In one implementation, the tooth position information of each tooth includes at least one of a tooth position number and a tooth key point of each tooth.

[0144] In one implementation, it also includes a geometric information map acquisition module, which is used to obtain the geometric information of the three-dimensional points in the first tooth model; the geometric information of the three-dimensional points is encoded and projected according to multiple different projection perspectives to obtain multiple first geometric information maps; the multiple first geometric information maps correspond one-to-one to the multiple first tooth images.

[0145] The tooth abnormality identification module 1102 includes:

[0146] The target image acquisition unit is used to superimpose each first tooth image and a first geometric information map corresponding to the first tooth image into a target image.

[0147] The tooth abnormality part recognition unit is used to input multiple target images into the tooth abnormality part recognition model so that the tooth abnormality part recognition model performs tooth abnormality part recognition processing on the multiple target images to obtain abnormal part recognition results corresponding to the multiple target images respectively.

[0148] In one implementation, the tooth abnormal part recognition unit is specifically used to input multiple target images into a tooth abnormal part recognition model, so that the tooth abnormal part recognition model extracts color texture features and geometric features from each target image, determines each part in the oral environment by analyzing the geometric features, and classifies the parts based on the color texture features and geometric features corresponding to each part in the oral environment to obtain abnormal part recognition results corresponding to each target image; wherein, each part in the oral environment includes at least one of the occlusal surface of the tooth, the non-occlusal surface of the tooth, the junction of the tooth and the gum, the gum, the labial side and the lingual side.

[0149] In one implementation, the tooth abnormal part identification unit is specifically used for: if the part is the occlusal surface of the tooth, the color texture feature indication corresponding to the part belongs to the dark range and the corresponding geometric feature of the part shows a changing trend, it is determined that there is a wedge-shaped defect at the part; if the part is the junction of the tooth and the gum, the color texture feature indication corresponding to the part belongs to the light range and the corresponding geometric feature of the part shows a changing trend, it is determined that there is tartar at the part; if the part is the non-occlusal surface, labial side or lingual side of the tooth, the color texture feature indication corresponding to the part belongs to the dark range and the corresponding geometric feature of the part shows a stable trend, it is determined that there is pigmentation at the part; if the part is the non-occlusal surface, labial side or lingual side of the tooth, the color texture feature indication corresponding to the part belongs to the dark range and the corresponding geometric feature of the part shows a changing trend, it is determined that there is caries at the part; if the part is the gum, the color texture feature indication corresponding to the part belongs to the red range and the corresponding geometric feature of the part shows a changing trend, it is determined that there is redness and swelling of the gums or gum atrophy at the part.

[0150] In one implementation, the three-dimensional points in the first tooth model include at least two types of geometric information.

[0151] The geometric information map acquisition module is specifically used for: for each type of geometric information of the three-dimensional point, rescaling the geometric information of this type to within a preset numerical range, and projecting the rescaled geometric information according to multiple different projection perspectives to obtain multiple first images, and respectively mapping the multiple first images to one of the color channels of the specified color mode to obtain multiple second images; wherein different types of geometric information of the three-dimensional point correspond to different color channels; the preset numerical range is the color representation range of the specified color mode; after obtaining multiple second images corresponding to all types of geometric information of the three-dimensional point, the second images belonging to the same projection perspective are fused to obtain multiple first geometric information maps.

[0152] In one implementation, the geometric information of the 3D point includes at least one of the following types: curvature, depth and normal vector of the 3D point. The target image is represented by at least four channels, three of which are used to describe the first tooth image and the remaining at least one channel is used to describe the first geometric information map.

[0153] In one implementation, the abnormal part recognition result corresponding to each first tooth image includes at least one graphic frame for indicating the abnormal part superimposed and displayed on the first tooth image;

[0154] The target abnormal part determination module 1103 is specifically used to back-project the abnormal part identification results of each first tooth image in the multiple first tooth images to present a graphic box for indicating the abnormal part in the first tooth model; count the number of graphic boxes for indicating the same abnormal part presented in the first tooth model; if the number of graphic boxes for indicating the same abnormal part presented in the first tooth model exceeds a preset number, determine that the abnormal part is the target abnormal part; wherein the preset number is greater than half of the total number of multiple different projection viewing angles.

[0155] In one implementation, the abnormal part recognition results corresponding to each first tooth image also include the abnormal type corresponding to each graphic box; wherein the abnormal type includes at least one of wedge-shaped defects, tartar, pigmentation, caries, gingival redness and swelling, and gingival atrophy.

[0156] The target abnormal part determination module 1103 is specifically used to determine that the abnormal part is the target abnormal part if the number of graphic boxes indicating the same abnormal part presented in the first tooth model exceeds a preset number and the abnormal types corresponding to all graphic boxes indicating the same abnormal part are the same abnormal type.

[0157] In one implementation, it also includes a screenshot module, which is used to determine the non-obstructed viewing angle of the target abnormal part on the first tooth model; wherein the non-obstructed viewing angle is a viewing angle at which the target abnormal part can be observed on the first tooth model; when the first tooth model is displayed on the display interface, the non-obstructed viewing angle is used as the screenshot viewing angle of the display interface, and the display interface is screenshot processed to obtain a two-dimensional screenshot containing the target abnormal part.

[0158] In some embodiments, see Fig.12 The present application also provides a training device for a tooth abnormality part recognition model, including:

[0159] The training sample acquisition module 1201 is used to acquire a number of training samples, wherein the training samples include a plurality of second tooth images of the second tooth model under a plurality of different projection perspectives and an abnormal part label corresponding to each second tooth image; in the plurality of second tooth images, each tooth in the second tooth model has projection results under at least two projection perspectives; different training samples correspond to different second tooth models.

[0160] The model training module 1202 is used to input the training samples into the tooth abnormality part recognition model to be trained, so that the tooth abnormality part recognition model to be trained recognizes the tooth abnormality part based on multiple second tooth images, obtains the abnormality part recognition results corresponding to the multiple second tooth images respectively, and trains the tooth abnormality part recognition model to be trained with the optimization goal of minimizing the difference between the abnormality part recognition results corresponding to each second tooth image and the abnormality part label corresponding to the second tooth image; wherein the trained tooth abnormality part recognition model is applied to the tooth abnormality part recognition device as mentioned above.

[0161] In one implementation, the training samples also include multiple second geometric information maps of the geometric information of the three-dimensional points in the second dental model under multiple different projection perspectives; the multiple second geometric information maps correspond one-to-one to the multiple second dental images; wherein the dental abnormality part recognition model to be trained is used to identify abnormal parts based on multiple target images; wherein each target image is obtained by superimposing each second dental image and the second geometric information map corresponding to the second dental image.

[0162] In one implementation, the training sample acquisition module 1201 is specifically used to: display the second dental model corresponding to each training sample in a display interface; in response to a labeling operation on the second dental model, add a marking point at the labeling position indicated by the labeling operation to obtain a second dental model carrying the marking point; project the second dental model carrying the marking point from a three-dimensional space to a two-dimensional space according to different projection angles to obtain multiple second dental images carrying the marking point; generate a graphic box of a preset size for indicating an abnormal part with the marking point in each second dental image carrying the marking point as the center, and determine the graphic box as the abnormal part label corresponding to the second dental image.

[0163] In one implementation, the abnormal part label corresponding to each second tooth image also includes an abnormality type corresponding to each graphic box; the abnormality type includes at least one of wedge-shaped defects, tartar, pigmentation, gingival redness and swelling, and gingival atrophy; the abnormality type corresponding to each graphic box is obtained based on the labeling operation.

[0164] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are only illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units.

[0165] In some embodiments, the embodiments of the present application further provide an electronic device, including a memory, a processor, and executable instructions stored in the memory and executable on the processor;

[0166] Wherein, the processor implements the steps in the above method when executing the executable instructions.

[0167] Exemplarily, the processor includes but is not limited to a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), or a field-programmable gate array (FPGA).

[0168] Exemplarily, the memory may include at least one type of storage medium, the storage medium including flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc.

[0169] For example, Fig.13 is a schematic structural diagram of a device provided by an exemplary embodiment. Fig.13 At the hardware level, the device includes a processor 1302, an internal bus 1304, a network interface 1306, a memory 1308, and a non-volatile memory 1310, and may also include hardware required for other services. One or more embodiments of this specification may be implemented based on software, such as the processor 1302 reading the corresponding computer program from the non-volatile memory 1310 into the memory 1308 and then running it. Of course, in addition to the software implementation, one or more embodiments of this specification do not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0170] In some embodiments, the embodiments of the present application further provide a computer program product, including a computer program, which is used to implement the above method when executed by a processor.

[0171] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory including instructions, and the above instructions can be executed by a processor of the device to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0172] A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a processor of a terminal, enables the terminal to execute the above method.

[0173] Although this specification includes many specific implementation details, these should not be interpreted as limiting the scope of any invention or the scope of protection claimed, but are mainly used to describe the features of the specific embodiments of specific inventions. Certain features described in multiple embodiments in this specification may also be implemented in combination in a single embodiment. On the other hand, the various features described in a single embodiment may also be implemented separately in multiple embodiments or in any suitable sub-combination. In addition, although features may work in certain combinations as described above and even initially claim protection, one or more features from the claimed combination may be removed from the combination in some cases, and the claimed combination may point to a sub-combination or a variation of a sub-combination.

[0174] Similarly, although operations are depicted in a particular order in the accompanying drawings, this should not be understood as requiring that these operations be performed in the particular order shown or performed sequentially, or requiring that all illustrated operations be performed to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system modules and components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product, or packaged into multiple software products.

[0175] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for identifying abnormal parts of teeth, characterized in that: include: The first tooth model to be identified is projected according to a plurality of different projection viewing angles to obtain a plurality of first tooth images; wherein the plurality of different projection viewing angles are used to make each tooth in the first tooth model have projection results under at least two projection viewing angles in the plurality of first tooth images; and Acquire geometric information of three-dimensional points in the first tooth model; encode and project the geometric information of the three-dimensional points according to the multiple different projection perspectives to obtain multiple first geometric information maps; the multiple first geometric information maps correspond one-to-one to the multiple first tooth images; wherein the geometric information of the three-dimensional points in the first tooth model is used to provide information about spatial structure and shape characteristics; superimposing each first tooth image and a first geometric information map corresponding to the first tooth image into a target image; Input the plurality of target images into the tooth abnormality recognition model, so that the tooth abnormality recognition model performs tooth abnormality recognition processing on the plurality of target images, and obtains abnormality recognition results corresponding to the plurality of target images respectively; wherein the tooth abnormality recognition model is obtained by supervised training based on a plurality of training samples, and the training samples include a plurality of second geometric information graphs of geometric information of three-dimensional points in the second tooth model under the plurality of different projection perspectives, a plurality of second tooth images of the second tooth model under the plurality of different projection perspectives, and an abnormality label corresponding to each of the second tooth images; the plurality of second geometric information graphs correspond one-to-one to the plurality of second tooth images; Screening the abnormal part recognition results corresponding to the plurality of target images to determine the target abnormal part; The three-dimensional points in the first tooth model include at least two types of geometric information; the geometric information of the three-dimensional points is encoded and projected according to the multiple different projection perspectives to obtain multiple first geometric information maps, including: For each type of geometric information of the three-dimensional point, rescale the geometric information of this type to a preset value range, and project the rescaled geometric information according to the multiple different projection viewing angles to obtain multiple first images, and respectively map the multiple first images to one of the color channels of the specified color mode to obtain multiple second images; wherein the different types of geometric information of the three-dimensional point correspond to different color channels; and the preset value range is the color representation range of the specified color mode; After obtaining the plurality of second images respectively corresponding to all types of geometric information of the three-dimensional point, the second images belonging to the same projection viewing angle are fused to obtain the plurality of first geometric information images.

2. The method according to claim 1, characterized in that The projecting the first tooth model to be identified according to a plurality of different projection viewing angles to obtain a plurality of first tooth images includes: Performing tooth position recognition on each tooth in the first tooth model to determine tooth position information of each tooth in the first tooth model; Based on the tooth position information of each tooth in the first tooth model and the tooth position information of each tooth in the predetermined template tooth model, aligning the first tooth model with the template tooth model to obtain an aligned first tooth model; The aligned first tooth model is projected according to a plurality of different projection viewing angles preset corresponding to the template tooth model to obtain the plurality of first tooth images.

3. The method according to claim 2, characterized in that The tooth position information of each tooth includes at least one of a tooth position number and a tooth key point of each tooth.

4. The method according to claim 1, characterized in that: The step of inputting the plurality of target images into the tooth abnormality recognition model so as to perform tooth abnormality recognition processing on the plurality of target images by the tooth abnormality recognition model to obtain abnormality recognition results respectively corresponding to the plurality of target images comprises: Input a plurality of the target images into the tooth abnormal part recognition model, so that the tooth abnormal part recognition model extracts color texture features and geometric features from each of the target images, determines each part in the oral environment by analyzing the geometric features, and classifies the parts based on the color texture features and geometric features corresponding to each part in the oral environment to obtain abnormal part recognition results corresponding to each of the target images; wherein the various parts in the oral environment include at least one of the occlusal surface of the tooth, the non-occlusal surface of the tooth, the junction between the tooth and the gum, the gum, the labial side and the lingual side.

5. The method according to claim 4, characterized in that The classification based on the color texture features and geometric features corresponding to each part in the oral environment includes: If the part is the occlusal surface of the tooth, the color texture feature corresponding to the part indicates that it belongs to the dark range, and the geometric feature corresponding to the part shows a changing trend, it is determined that there is a wedge-shaped defect at the part; If the part is the junction of the teeth and gums, the color texture feature corresponding to the part indicates that it belongs to the light color range, and the geometric feature corresponding to the part shows a changing trend, it is determined that dental calculus exists in the part; If the part is the non-occlusal surface, labial side or lingual side of the tooth, the color texture feature corresponding to the part indicates that it belongs to the dark range, and the geometric feature corresponding to the part shows a stable trend, it is determined that pigmentation exists in the part; If the part is the non-occlusal surface, labial side or lingual side of the tooth, the color texture feature corresponding to the part indicates that it belongs to the dark range, and the geometric feature corresponding to the part shows a changing trend, it is determined that caries exists at the part; If the part is the gums, the color texture feature corresponding to the part indicates that it belongs to the red range, and the geometric feature corresponding to the part shows a changing trend, it is determined that red and swollen gums or gum atrophy exist in the part.

6. The method according to any one of claims 1 to 5, characterized in that: The geometric information of the three-dimensional point includes at least one of the following types: curvature, depth and normal vector of the three-dimensional point; and / or The target image is represented by at least four channels, three of which are used to describe the first tooth image, and the remaining at least one channel is used to describe the first geometric information map.

7. The method according to claim 1, characterized in that The abnormal part recognition result corresponding to each of the target images includes at least one graphic frame for indicating the abnormal part superimposed and displayed on the target image; The screening process of the abnormal part recognition results corresponding to the plurality of target images to determine the target abnormal part includes: Back-projecting the abnormal part recognition result of each of the plurality of target images to present a graphic frame indicating the abnormal part in the first tooth model; Counting the number of graphic frames indicating the same abnormal part presented in the first tooth model; If the number of graphic boxes indicating the same abnormal part presented in the first tooth model exceeds a preset number, the abnormal part is determined to be a target abnormal part; wherein the preset number is greater than half of the total number of the multiple different projection viewing angles.

8. The method according to claim 7, characterized in that The abnormal part recognition results corresponding to each of the target images further include the abnormality type corresponding to each of the graphic frames; wherein the abnormality type includes at least one of wedge-shaped defect, dental calculus, pigmentation, caries, gingival redness and swelling, and gingival atrophy; If the number of graphic frames indicating the same abnormal part presented in the first tooth model exceeds a preset number, determining the abnormal part as a target abnormal part comprises: If the number of graphic boxes indicating the same abnormal part presented in the first tooth model exceeds a preset number, and the abnormal types corresponding to all graphic boxes indicating the same abnormal part are the same abnormal type, the abnormal part is determined to be a target abnormal part.

9. The method according to claim 7 or 8, characterized in that: Also includes: Determine a non-blocked viewing angle of the target abnormal part on the first tooth model; wherein the non-blocked viewing angle is a viewing angle at which the target abnormal part can be observed on the first tooth model; When the first tooth model is displayed on the display interface, the non-blocked viewing angle is used as the screenshot viewing angle of the display interface, and the display interface is screenshot processed to obtain a two-dimensional screenshot including the target abnormal part.

10. A method for training a tooth abnormality recognition model, characterized in that: include: Acquire a plurality of training samples, wherein the training samples include a plurality of second tooth images of the second tooth model at a plurality of different projection viewing angles, a plurality of second geometric information graphs of geometric information of three-dimensional points in the second tooth model at the plurality of different projection viewing angles, and an abnormal part label corresponding to each of the second tooth images; In the plurality of second tooth images, each tooth in the second tooth model has projection results under at least two projection viewing angles; The plurality of second geometric information graphs correspond one-to-one to the plurality of second tooth images; Different training samples correspond to different second tooth models; The training samples are input into a tooth abnormality part recognition model to be trained, so that the tooth abnormality part recognition model to be trained recognizes the tooth abnormality part based on the multiple second tooth images, obtains abnormality part recognition results corresponding to the multiple second tooth images respectively, and takes minimizing the difference between the abnormality part recognition results corresponding to each second tooth image and the abnormality part label corresponding to the second tooth image as the optimization goal, and trains the tooth abnormality part recognition model to be trained; wherein the tooth abnormality part recognition model to be trained is used to recognize the abnormality part based on multiple target images; each target image is obtained by superimposing each second tooth image and the second geometric information graph corresponding to the second tooth image; Wherein, the trained tooth abnormality part recognition model is applied to the tooth abnormality part recognition method as described in any one of claims 1 to 9.

11. The method according to claim 10, characterized in that Acquiring the abnormal part label corresponding to each second tooth image in the training sample includes: Displaying the second tooth model corresponding to each training sample in the display interface; In response to the marking operation on the second tooth model, adding a marking point at the marking position indicated by the marking operation to obtain a second tooth model carrying the marking point; Projecting the second tooth model carrying the marking points from the three-dimensional space to the two-dimensional space according to the different projection viewing angles to obtain a plurality of second tooth images carrying the marking points; A graphic frame of a preset size for indicating an abnormal part is generated with the marked point in each second tooth image carrying the marked point as the center, and the graphic frame is determined as the abnormal part label corresponding to the second tooth image.

12. A device for identifying abnormal parts of teeth, characterized in that: include: A first tooth image acquisition module, used for projecting the first tooth model to be identified according to a plurality of different projection viewing angles to obtain a plurality of first tooth images; wherein the plurality of different projection viewing angles are used to make each tooth in the first tooth model have projection results under at least two projection viewing angles in the plurality of first tooth images; A geometric information map acquisition module is used to acquire geometric information of three-dimensional points in the first tooth model; the geometric information of the three-dimensional points is encoded and projected according to a plurality of different projection angles to obtain a plurality of first geometric information maps; the plurality of first geometric information maps correspond one-to-one to a plurality of first tooth images; wherein the geometric information of the three-dimensional points in the first tooth model is used to provide information about spatial structure and shape characteristics; A tooth abnormality part recognition module is used to perform tooth abnormality part recognition processing on the multiple first tooth images using a pre-trained tooth abnormality part recognition model to obtain abnormality part recognition results corresponding to the multiple first tooth images respectively; wherein the tooth abnormality part recognition module includes: a target image acquisition unit, used to superimpose each first tooth image and a first geometric information map corresponding to the first tooth image into a target image; a tooth abnormality part recognition unit, used to input multiple target images into the tooth abnormality part recognition model, so that the tooth abnormality part recognition model performs tooth abnormality part recognition processing on the multiple target images to obtain abnormality part recognition results corresponding to the multiple target images respectively; the tooth abnormality part recognition model is obtained by supervised training based on a number of training samples, and the training samples include a plurality of second geometric information maps of geometric information of three-dimensional points in the second tooth model at the multiple different projection angles, a plurality of second tooth images of the second tooth model at the multiple different projection angles, and an abnormal part label corresponding to each of the second tooth images; the multiple second geometric information maps correspond one-to-one to the multiple second tooth images; A target image determination module, used for screening the abnormal part recognition results corresponding to the plurality of target images to determine the target abnormal part; Among them, the three-dimensional points in the first tooth model include at least two types of geometric information; the geometric information map acquisition module is specifically used to rescale the geometric information of each type of geometric information of the three-dimensional points to within a preset numerical range, and project the rescaled geometric information according to the multiple different projection perspectives to obtain multiple first images, and respectively map the multiple first images to one of the color channels of the specified color mode to obtain multiple second images; wherein different types of geometric information of the three-dimensional points correspond to different color channels; the preset numerical range is the color representation range of the specified color mode; after obtaining the multiple second images corresponding to all types of geometric information of the three-dimensional points, the second images belonging to the same projection perspective are fused to obtain the multiple first geometric information maps.

13. An electronic device, characterized in that: including a memory, a processor, and executable instructions stored in the memory and executable on the processor; Wherein, when the processor executes the executable instructions, the steps in the method as described in any one of claims 1 to 11 are implemented.

14. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

Images