Method, system and device for obtaining three-dimensional oral multi-faceted model

Abstract

The present invention discloses a method, system and device for acquiring a three-dimensional oral multi-faceted model, and relates to the technical field of image processing. The method comprises comparing a shooting state coefficient with a preset shooting state coefficient range, and comparing the actual brightness of an LED cold light lamp with a preset brightness range, to obtain image data, and then performing image processing on the image data to obtain an available image set. In this way, it is determined whether the acquired intraoral image meets the requirements, and the influence of the final converted three-dimensional model being inaccurate due to the unavailability of the acquired intraoral image is reduced. The initial three-dimensional model is obtained by extracting key nodes from the available image set and calculating the image distance of the key nodes. The key characteristic values ​​are then copied to the initial three-dimensional model to obtain the final three-dimensional model. In this way, a two-dimensional image is analyzed in many aspects, and the influence of the final converted three-dimensional model being inaccurate due to the failure to analyze the two-dimensional image in many aspects is reduced.

Description

Technical Field

[0001] The present invention belongs to the field of image processing, and in particular relates to a method, system and device for acquiring a three-dimensional oral multi-faceted model. Background Art

[0002] Oral health is receiving more and more attention and importance. With the advancement of medical technology and the popularization of health education, more and more people are aware of the close relationship between oral health and overall health. A healthy oral cavity can not only improve people's quality of life, but also help prevent a variety of systemic diseases. Now, people's attention to oral health is not only limited to daily brushing, flossing and regular teeth cleaning. More and more people choose to go to the hospital for diagnosis and treatment when they have dental problems, in order to achieve more perfect oral health and a beautiful smile.

[0003] Currently, when doctors perform oral diagnosis on patients, in order to better understand the situation inside the patient's mouth, they will take pictures of the inside of the patient's mouth before treatment, obtain multiple two-dimensional image local framing pictures, and then convert the multiple two-dimensional image local framing pictures into a three-dimensional model. In this way, doctors can better observe the patient's mouth as a whole and reduce treatment omissions or diagnostic omissions.

[0004] However, in the above-mentioned technology, the conversion into a three-dimensional model is basically performed directly based on the partial framing view of the two-dimensional image, without judging whether the multiple partial framing views of the two-dimensional images taken are available, which may cause the final converted three-dimensional model to be inaccurate; in addition, in the process of converting the partial framing view of the two-dimensional image into a three-dimensional model, the partial framing view of the two-dimensional image is not analyzed in many aspects, resulting in the final converted three-dimensional model to be inaccurate. Summary of the invention

[0005] The purpose of the present invention is to solve the problem that whether the local viewpoints of multiple two-dimensional images taken are usable is not determined, which may cause the final converted three-dimensional model to be not accurate enough; in addition, in the process of converting the local viewpoints of the two-dimensional image into a three-dimensional model, only the coordinates of some nodes in the local viewpoints of the two-dimensional image are simply collected for three-dimensional conversion, and the local viewpoints of the two-dimensional image are not analyzed in many aspects, resulting in the final converted three-dimensional model being not accurate enough. A method, system and device for obtaining a three-dimensional oral multi-faceted model are proposed.

[0006] In a first aspect of the present invention, a method for obtaining a three-dimensional oral multi-faceted model is first proposed, the method comprising:

[0007] Acquire the operating status data of the camera module when shooting the internal image of the oral cavity, and obtain the shooting status coefficient according to the operating status data; the operating status data includes the angle rotation deviation coefficient and the voltage low-frequency coefficient;

[0008] Compare the shooting state coefficient with a preset shooting state coefficient range, compare the actual brightness of the LED cold light with a preset brightness range, and output a set of images shot by the camera module according to the comparison result to obtain image data;

[0009] Performing image processing on the image data to obtain a usable image set, extracting key nodes from the usable image set, calculating image distances of all the key nodes to obtain a total image distance set, obtaining precise three-dimensional coordinates of each key node based on the total image distance set, and mapping the precise three-dimensional coordinates to a three-dimensional module to obtain an initial three-dimensional model;

[0010] The key feature values ​​in the available image set are captured, and the key feature values ​​are copied to the initial three-dimensional model to obtain a final three-dimensional model.

[0011] Optionally, the operating state data of the camera module when shooting the internal image of the oral cavity is obtained, and the shooting state coefficient is obtained according to the operating state data, including:

[0012] Obtaining an actual rotation angle and a preset rotation angle in a fixed time period when the camera module shoots inside the patient's oral cavity within a fixed time interval, and calculating the actual rotation angle and the preset rotation angle to obtain an angle rotation deviation coefficient;

[0013] Obtaining the real-time voltage and the preset minimum voltage flowing through the camera module in a fixed time period when the camera module is shooting inside the patient's oral cavity within a fixed time interval, and calculating the real-time voltage and the preset minimum voltage to obtain a voltage low-frequency coefficient;

[0014] The angle rotation deviation coefficient and the voltage low-frequency coefficient are weightedly summed to obtain a shooting state coefficient.

[0015] Optionally, comparing the shooting state coefficient with a preset shooting state coefficient range includes:

[0016] If the shooting state coefficient is within the preset shooting state coefficient range, it is recorded as a normal shooting state;

[0017] If the shooting state coefficient is not within the preset shooting state coefficient range, it is recorded as abnormal shooting state;

[0018] Comparison between the actual brightness of the LED cold light and the preset brightness range includes:

[0019] If the actual brightness of the LED cold light lamp is within the preset brightness range, it is recorded as the brightness of the LED cold light lamp is normal;

[0020] If the actual brightness of the LED cold light lamp is within the preset brightness range, it is recorded as abnormal brightness of the LED cold light lamp;

[0021] When the shooting state is normal and the brightness of the LED cold light lamp is normal, the camera module captures the image set and obtains the image data; otherwise, the camera module and the brightness state of the LED cold light lamp are adjusted, and the image inside the oral cavity is captured again until the shooting state is normal and the brightness of the LED cold light lamp is normal, and the image data is output.

[0022] Optionally, performing image processing on the image data to obtain a usable image set includes:

[0023] Sorting the image data according to shooting time to obtain a sequence image set;

[0024] For each pair of consecutive images in the image sequence, the optical flow vector between the two frames is calculated;

[0025] For each sequence of images, the pixel position is corrected according to the optical flow vector, and at the new pixel position, bilinear interpolation is used to estimate the pixel value at the new position to obtain a usable image set.

[0026] Optionally, extracting key nodes from the available image set, calculating image distances of all key nodes, and obtaining an image distance set includes:

[0027] Extract all key nodes in the available image set, and calculate the distances of all key nodes in the image containing the key nodes to obtain an image distance set for each key node, and further aggregate the image distance set for each key node to obtain an overall image distance set;

[0028] And according to the overall image distance set, the precise three-dimensional coordinates of each key node are obtained, and the precise three-dimensional coordinates are mapped to the three-dimensional module to obtain the initial three-dimensional model including:

[0029] Selecting a reference three-dimensional node in the patient's oral cavity, and assigning the reference three-dimensional node to a preset three-dimensional coordinate, obtaining a basic three-dimensional model, and calculating a three-dimensional distance of the reference three-dimensional node in the basic three-dimensional model;

[0030] Obtaining a corresponding node in a preset image corresponding to the reference three-dimensional node in the available image set to obtain a marked node, and calculating a two-dimensional image distance of the marked node to obtain a two-dimensional distance of the marked node;

[0031] Calculating the ratio of the two-dimensional distance of the marked node to the three-dimensional distance of the reference three-dimensional node to obtain a basic ratio;

[0032] Obtaining the camera module angle corresponding to the basic ratio to obtain an initial angle, extracting the image distance set of each group of key nodes in the image distance set, and obtaining the angle of rotation of the camera module corresponding to different data values ​​in the image distance set of the key node relative to the initial angle to obtain the corresponding actual angle;

[0033] According to the data value, the basic ratio and the actual angle, the three-dimensional coordinates of the key node corresponding to each data value in the image distance set of the key node are calculated to obtain the three-dimensional coordinate set of the key node;

[0034] The coordinates in the three-dimensional coordinate set of the key node are averaged to obtain the precise coordinates, and the image distance set corresponding to all the key nodes in the overall image distance set is averaged according to the above steps to obtain the precise coordinates corresponding to each key node, and all the precise coordinates are mapped to the three-dimensional module to obtain the initial three-dimensional model.

[0035] Optionally, capturing key feature values ​​in the available image set and copying the key feature values ​​to the initial three-dimensional model to obtain the final three-dimensional model includes:

[0036] The contours of tooth crests, troughs and gap points in the available image set are obtained to obtain key feature values, and the key feature values ​​in the available image set are captured and copied to the initial three-dimensional model to obtain the final three-dimensional model.

[0037] In a second aspect of the present invention, a system for acquiring a three-dimensional oral multi-faceted model is proposed, the system comprising a camera module, an LED cold light lamp and a back-end analysis device, the LED cold light lamp emits different brightness in real time to supplement the camera module with the image of the inside of the oral cavity, wherein:

[0038] The operating state data of the camera module when taking images of the interior of the oral cavity, and obtaining a shooting state coefficient according to the operating state data; the operating state data includes an angle rotation deviation coefficient and a voltage low-frequency coefficient;

[0039] The back-end analysis device compares the shooting state coefficient with a preset shooting state coefficient range, compares the actual brightness of the LED cold light with a preset brightness range, and outputs an image set shot by the camera module according to the comparison result to obtain image data;

[0040] The back-end analysis device performs image processing on the image data to obtain a usable image set, extracts key nodes from the usable image set, calculates image distances of all key nodes to obtain a total image distance set, obtains precise three-dimensional coordinates of each key node based on the total image distance set, and maps the precise three-dimensional coordinates to a three-dimensional module to obtain an initial three-dimensional model;

[0041] The back-end analysis device captures key feature values ​​from the available image set, and copies the key feature values ​​to the initial three-dimensional model to obtain a final three-dimensional model.

[0042] In a third aspect of the present invention, a device for obtaining a three-dimensional oral multi-faceted model is provided, the device comprising:

[0043] Shooting status module: obtains the operating status data of the camera module when shooting the internal image of the oral cavity, and obtains the shooting status coefficient according to the operating status data; the operating status data includes the angle rotation deviation coefficient and the voltage low-frequency coefficient;

[0044] Image data module: compares the shooting state coefficient with a preset shooting state coefficient range, compares the actual brightness of the LED cold light with a preset brightness range, and outputs a set of images shot by the camera module according to the comparison result to obtain image data;

[0045] Initial 3D model module: performing image processing on the image data to obtain a usable image set, extracting key nodes from the usable image set, calculating image distances of all key nodes to obtain a total image distance set, obtaining precise 3D coordinates of each key node based on the total image distance set, and mapping the precise 3D coordinates to the 3D module to obtain an initial 3D model;

[0046] Final three-dimensional model module: captures key feature values ​​from the available image set and copies the key feature values ​​to the initial three-dimensional model to obtain a final three-dimensional model.

[0047] Beneficial effects of the present invention:

[0048] The present invention proposes a method, system and device for obtaining a three-dimensional oral multi-faceted model, by comparing a shooting state coefficient with a preset shooting state coefficient range, and comparing the actual brightness of an LED cold light with a preset brightness range, and outputting an image set shot by a camera module according to the comparison result to obtain image data, and then performing image processing on the image data to obtain a usable image set. In this way, it is determined that the acquired intraoral image meets the subsequent three-dimensional modeling requirements, thereby reducing the impact of the final converted three-dimensional model being inaccurate due to the unavailability of the acquired intraoral image; by extracting key nodes from the usable image set, calculating the image distance of the key nodes of each image, and mapping the image distance to the three-dimensional module, an initial three-dimensional model is obtained; and by capturing key feature values ​​in the usable image set, and copying the key feature values ​​to the initial three-dimensional model, a final three-dimensional model is obtained. In this way, the acquired two-dimensional image is analyzed in many aspects, thereby reducing the impact of the final converted three-dimensional model being inaccurate due to the failure to analyze the two-dimensional image in many aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The present invention will be further described below in conjunction with the accompanying drawings.

[0050] Figure 1 A flowchart of a method for obtaining a three-dimensional oral multi-faceted model provided by an embodiment of the present invention;

[0051] Figure 2 A framework diagram of a system for acquiring a three-dimensional oral multi-faceted model provided by an embodiment of the present invention;

[0052] Figure 3 A framework diagram of a device for acquiring a three-dimensional oral multi-faceted model provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0053] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without making any creative work shall fall within the scope of protection of the present invention.

[0054] The embodiment of the present invention provides a method for obtaining a three-dimensional oral multi-faceted model. Figure 1 , Figure 1 A flow chart of a method for obtaining a three-dimensional oral multi-faceted model provided in an embodiment of the present invention. The method comprises:

[0055] Obtaining the operating status data of the camera module when shooting the internal image of the oral cavity, and obtaining the shooting status coefficient according to the operating status data;

[0056] Compare the shooting state coefficient with the preset shooting state coefficient range, compare the actual brightness of the LED cold light with the preset brightness range, and output the image set shot by the camera module according to the comparison result to obtain image data;

[0057] Perform image processing on the image data to obtain a usable image set, extract key nodes from the usable image set, calculate the image distances of all key nodes to obtain a total image distance set, obtain the precise three-dimensional coordinates of each key node based on the total image distance set, and map the precise three-dimensional coordinates to a three-dimensional module to obtain an initial three-dimensional model;

[0058] The key feature values ​​of the available image set are captured and copied to the initial 3D model to obtain the final 3D model.

[0059] In one implementation method, by comparing the shooting state coefficient with the preset shooting state coefficient range, and comparing the actual brightness of the LED cold light with the preset brightness range, and outputting the image set shot by the camera module according to the comparison result, image data is obtained, and then the image data is processed to obtain a usable image set. In this way, it is determined that the acquired intraoral image meets the subsequent three-dimensional modeling requirements, reducing the impact of the final converted three-dimensional model being inaccurate due to the unavailability of the acquired intraoral image; by extracting the key nodes in the available image set, calculating the image distance of the key nodes of each image, and mapping the image distance to the three-dimensional module, an initial three-dimensional model is obtained; the key feature values ​​in the available image set are captured, and the key feature values ​​are copied to the initial three-dimensional model to obtain the final three-dimensional model. In this way, the acquired two-dimensional image is analyzed in many aspects, reducing the impact of the final converted three-dimensional model being inaccurate due to the unilateral analysis of the two-dimensional image.

[0060] It should be noted that the LED cold light is installed around the camera module. When the camera module enters the patient's mouth to shoot, the soft tissues in the mouth, such as the tongue and oral mucosa, may reflect red light, which may cause over-saturated red in the image, thereby affecting the contrast and details of the image, and causing the captured image to drift; similarly, the surface of the teeth in the mouth is smooth and may reflect ambient light or the flash of the camera, which may cause white spots or reflective areas in the image, affecting the clarity and accuracy of the image, and also causing the captured image to drift; by installing LED cold light around the camera module and adjusting the brightness of the LED cold light in real time, the lighting conditions inside the mouth can be changed, thereby affecting the intensity and direction of the reflected light on the surface of the teeth and soft tissues. Appropriately adjusting the brightness of the LED cold light can reduce the image drift caused by the reflected light from the teeth and soft tissues, and improve the clarity and accuracy of the oral image.

[0061] In one embodiment, the operating state data of the camera module when shooting the internal image of the oral cavity is obtained, and the shooting state coefficient is obtained according to the operating state data, including:

[0062] Obtain the actual rotation angle and the preset rotation angle of the fixed time period when the camera module shoots inside the patient's mouth within a fixed time interval, and calculate the actual rotation angle and the preset rotation angle to obtain the angle rotation deviation coefficient, specifically including:

[0063] The actual rotation angle and the preset rotation angle of the camera module in a fixed time period within a fixed time interval are obtained, and they correspond one to one to obtain a data set. There are n data in the data set, and each data includes an actual rotation angle of the camera module and a corresponding preset rotation angle. The difference between the actual rotation angle and the corresponding preset rotation angle in each data is calculated, and the difference is compared with the preset threshold. If the difference is not less than the preset threshold, the data is marked as 0. If the difference is less than the preset threshold, the data is marked as 1. The number of 0s in the data set is calculated and recorded as m. The angle rotation deviation coefficient is calculated. The calculation expression is: FG=m / n, where FG is the angle rotation deviation coefficient;

[0064] Obtain the real-time voltage and the preset minimum voltage flowing through the camera module in a fixed time period when the camera module is shooting inside the patient's mouth within a fixed time interval, and calculate the real-time voltage and the preset minimum voltage to obtain the voltage low-frequency coefficient, specifically including:

[0065] Obtain the real-time voltage and the preset minimum voltage flowing through the camera module in a fixed time period when the camera module shoots inside the patient's mouth within a fixed time interval, and record the time period as h. Compare the real-time voltage flowing through the camera module with the preset minimum voltage. If the real-time voltage flowing through the camera module is less than the preset minimum voltage, record the time period as the acquisition error time period. Calculate the number of acquisition error time periods in all time periods and mark it as i. Calculate the voltage low-frequency coefficient. The calculation formula is: LK=i / h, where LK is the voltage low-frequency coefficient.

[0066] The shooting state coefficient is obtained by weighted summing the angle rotation deviation coefficient and the voltage low-frequency coefficient.

[0067] The calculation formula is: Where Yh is the shooting state coefficient.

[0068] In one implementation method, by obtaining the angle rotation deviation coefficient and the voltage low-frequency coefficient of the camera module, the shooting state coefficient of the camera module when taking images of the patient's oral cavity is judged from two directions, which can more accurately judge the shooting state of the camera module when taking images of the patient's oral cavity.

[0069] In one embodiment, comparing the shooting state coefficient with a preset shooting state coefficient range includes:

[0070] If the shooting state coefficient is within the preset shooting state coefficient range, it is recorded as normal shooting state;

[0071] If the shooting state coefficient is not within the preset shooting state coefficient range, it is recorded as abnormal shooting state;

[0072] Comparison between the actual brightness of the LED cold light and the preset brightness range includes:

[0073] If the actual brightness of the LED cold light lamp is within the preset brightness range, it is recorded as the brightness of the LED cold light lamp is normal;

[0074] If the actual brightness of the LED cold light lamp is within the preset brightness range, it is recorded as abnormal brightness of the LED cold light lamp;

[0075] When the shooting status is normal and the brightness of the LED cold light lamp is normal, the image set captured by the camera module is output to obtain image data; otherwise, the camera module and the brightness status of the LED cold light lamp are adjusted, and the image inside the oral cavity is captured again until the shooting status is normal and the brightness of the LED cold light lamp is normal, and the image data is output.

[0076] In one implementation, the preset shooting state coefficient range is a range determined by professionals according to actual conditions, and the preset brightness range is a range determined by professionals according to actual conditions; because when the camera module shoots the patient's oral condition, the camera module has a fixed rotation method to shoot the entire oral cavity, and because the brightness of the red light reflected by the soft tissue in the oral cavity and the white light reflected by the teeth may be different when the camera module is in different positions in the oral cavity, the image drift is reduced by adjusting the brightness of the LED cold light lamp in real time. Therefore, it is necessary to ensure that the real-time brightness of the LED cold light lamp is within the preset brightness range. Only when the shooting state is normal and the brightness of the LED cold light lamp is normal, the image set shot by the camera module is output to obtain image data;

[0077] If the shooting status is normal but the brightness of the LED cold light is abnormal, the shooting status is abnormal but the brightness of the LED cold light is normal, and the shooting status is abnormal and the brightness of the LED cold light is abnormal, it is necessary to adjust the camera module and the brightness status of the LED cold light again, and take the image of the inside of the oral cavity again until the shooting status is normal and the brightness of the LED cold light is normal, and output the image data. In this way, image drift is reduced to ensure that the acquired image has high availability.

[0078] In one embodiment, image processing is performed on the image data to obtain a usable image set including:

[0079] Sort the image data according to the shooting time to obtain a sequence image set;

[0080] For each pair of consecutive images in the image sequence, the optical flow vector between the two frames is calculated;

[0081] For each sequence of images, the pixel position is corrected according to the optical flow vector, and bilinear interpolation is used to estimate the pixel value at the new pixel position to obtain a usable image set.

[0082] In one implementation method, in the above steps, although the image drift when the camera module captures the image of the patient's mouth is greatly reduced by real-time adjustment of the brightness of the LED cold light lamp, some slight image drift may still exist due to the combined influence of various factors. Therefore, in order to minimize the impact of image drift, the image data inside the oral cavity can be effectively processed through the above method to minimize the slight drift and discontinuity in the image, thereby obtaining a set of clear, continuous and usable images, making the subsequent conversion into three-dimensional process more accurate.

[0083] In one embodiment, key nodes in the available image set are extracted, and image distances of all key nodes are calculated to obtain an image distance set including:

[0084] Extract all key nodes in the available image set, and calculate the distances of all key nodes in the image containing the key nodes to obtain the image distance set of each key node, and then aggregate the image distance set of each key node again to obtain the overall image distance set;

[0085] And according to the overall image distance set, the precise three-dimensional coordinates of each key node are obtained, and the precise three-dimensional coordinates are mapped to the three-dimensional module to obtain the initial three-dimensional model including:

[0086] Selecting a reference 3D node in the patient's oral cavity, assigning the reference 3D node to a preset 3D coordinate, obtaining a basic 3D model, and calculating a 3D distance of the reference 3D node in the basic 3D model;

[0087] Obtaining the corresponding node in the preset image corresponding to the reference 3D node in the available image set to obtain a marked node, and calculating the 2D image distance of the marked node to obtain the 2D distance of the marked node;

[0088] Calculate the ratio of the two-dimensional distance of the marked node to the three-dimensional distance of the reference three-dimensional node to obtain the basic ratio;

[0089] Obtain the camera module angle corresponding to the basic ratio to obtain the initial angle, extract the image distance set of each group of key nodes in the image distance set, and obtain the angle of rotation of the camera module corresponding to different data values ​​in the image distance set of the key node relative to the initial angle to obtain the corresponding actual angle;

[0090] According to the data value, the basic ratio and the actual angle, the three-dimensional coordinates of the key node corresponding to each data value in the image distance set of the key node are calculated to obtain the three-dimensional coordinate set of the key node;

[0091] The coordinates in the three-dimensional coordinate set of the key node are averaged to obtain the precise coordinates. The image distance set corresponding to all key nodes in the overall image distance set is averaged according to the above steps to obtain the precise coordinates corresponding to each key node. All precise coordinates are mapped to the three-dimensional module to obtain the initial three-dimensional model.

[0092] In one implementation, more accurate three-dimensional coordinates of key nodes are obtained in this way, and the relatively accurate three-dimensional coordinates of all key nodes are mapped to the three-dimensional module to obtain an initial three-dimensional model. In this way, the obtained initial three-dimensional model is more accurate.

[0093] It should be noted that the above steps are illustrated by examples:

[0094] For example, first determine the key nodes in the available image set, first determine a key node included in each image in the available image set, and record it as the two-dimensional origin;

[0095] Now, assuming that there is a key node A, calculate the distance from the key node A to the two-dimensional origin, which is recorded as the image distance. Assuming that there are a total of a images in the available image set that contain the key node A, calculate the distance from the key node A to the two-dimensional origin of different images in these a images, which is recorded as the image distance set;

[0096] Selecting a reference 3D node in the patient's oral cavity and assigning the reference 3D node to a preset 3D coordinate to obtain a basic 3D model, and calculating the 3D distance of the reference 3D node in the basic 3D model, wherein the origin in the basic 3D model is the 3D coordinate point corresponding to the 2D origin;

[0097] It should be noted that the selected reference three-dimensional node should correspond to a more conspicuous part in the actual oral cavity, which may be the middle gap between the upper teeth or the middle gap between the lower teeth; such a position is easy to identify and locate, and can usually be easily identified and located in different images and angles, which helps to accurately find the reference node in multiple images, thereby improving the accuracy and stability of the measurement; in addition, the gap between teeth or other conspicuous parts of the oral cavity are relatively stable anatomical landmarks, which are not prone to significant changes due to physiological changes or movement, which makes such reference nodes more suitable for use as reference points for three-dimensional models because they remain relatively stable at different times and conditions.

[0098] Extract the image with the most upper teeth (or lower teeth) in the available image set. Such images are generally taken in parallel. The image distance presented by the most middle tooth gap in the upper teeth or the most middle tooth gap in the lower teeth in the two-dimensional image is most consistent with the actual oral situation;

[0099] Then, a node corresponding to a certain reference 3D node in the reference 3D node in the above image is obtained to obtain a marked node, and then a distance from the marked node to the 2D origin in the image is calculated, which is recorded as the marked node 2D distance;

[0100] Calculate the ratio of the two-dimensional distance of the marked node to the three-dimensional distance of the reference three-dimensional node to obtain the basic ratio;

[0101] This basic ratio temporarily represents the ratio of the upper distance of the 2D image node to the distance in the 3D coordinates;

[0102] However, in order to obtain more accurate specific coordinates of nodes in the 3D model, it is necessary to combine data values ​​(image distances of key nodes in different images), basic ratios, and actual angles to obtain a more accurate coordinate position;

[0103] Obtain the camera module angle corresponding to the basic ratio to obtain the initial angle, extract each data value in the image distance set, and obtain the angle of rotation of the camera module corresponding to different data values ​​relative to the initial angle to obtain the corresponding actual angle;

[0104] Assume that for node A, all image distances are 2, 2.5, 3, and 3.5 respectively; the corresponding actual angles of the camera module are 30, 40, 50, and 60 respectively;

[0105] Different coordinates are obtained through each image distance, the corresponding actual angle of the camera module and the basic ratio. Assuming that the four coordinates are (2.1, 2.2, 3), (2.0, 2.1, 3.1), (2.1, 2.3, 2.9) and (1.9, 2.2, 3.2), the four coordinates are averaged to get a coordinate of (2.025, 2.2, 3.05). Then the more accurate coordinate of node A in the three-dimensional model is (2.025, 2.2, 3.05). In this way, the coordinates of all key nodes are calculated and corresponded one by one to the three-dimensional module to obtain the initial three-dimensional model. In this way, the initial three-dimensional model is more accurate.

[0106] In one embodiment, the key feature values ​​in the available image set are captured and copied to the initial three-dimensional model, and the final three-dimensional model is obtained including:

[0107] The contours of tooth crests, troughs and gap points in the available image set are obtained to obtain key feature values, and the key feature values ​​in the available image set are captured and copied to the initial three-dimensional model to obtain the final three-dimensional model.

[0108] In one implementation method, based on the initial three-dimensional model, the contours of the tooth peaks, troughs and gap points in the image set are extracted to obtain key feature values. The key feature values ​​in the available image set are captured and copied to the initial three-dimensional model. In this way, the obtained three-dimensional model can more accurately reflect the actual situation in the patient's mouth.

[0109] Based on the same inventive concept, the present invention also provides a system for acquiring a three-dimensional oral multi-faceted model. Figure 2 , Figure 2 The schematic diagram of the structure of the system for acquiring a three-dimensional oral multi-faceted model provided by an embodiment of the present invention includes a camera module, an LED cold light lamp, and a back-end analysis device. The LED cold light lamp emits different brightness in real time to supplement the camera module with the image of the inside of the oral cavity, wherein:

[0110] The operating state data of the camera module when taking images of the inside of the oral cavity, and the shooting state coefficient is obtained according to the operating state data; the operating state data includes an angle rotation deviation coefficient and a voltage low-frequency coefficient;

[0111] The back-end analysis device compares the shooting state coefficient with the preset shooting state coefficient range, and the actual brightness of the LED cold light with the preset brightness range, and outputs the image set shot by the camera module according to the comparison result to obtain image data;

[0112] The back-end analysis device processes the image data to obtain a usable image set, extracts key nodes from the usable image set, calculates the image distances of all key nodes, obtains an overall image distance set, obtains the precise three-dimensional coordinates of each key node based on the overall image distance set, and maps the precise three-dimensional coordinates to the three-dimensional module to obtain an initial three-dimensional model;

[0113] The back-end analysis device captures the key feature values ​​of the available image set and copies the key feature values ​​to the initial three-dimensional model to obtain the final three-dimensional model.

[0114] Based on the same inventive concept, the present invention also provides a device for obtaining a three-dimensional oral multi-faceted model. Figure 2 , Figure 2 A schematic diagram of the structure of a device for obtaining a three-dimensional oral multi-faceted model provided in an embodiment of the present invention, the device comprising:

[0115] Shooting status module: obtains the operating status data of the camera module when shooting the internal image of the oral cavity, and obtains the shooting status coefficient according to the operating status data; the operating status data includes the angle rotation deviation coefficient and the voltage low-frequency coefficient;

[0116] Image data module: compares the shooting state coefficient with the preset shooting state coefficient range, compares the actual brightness of the LED cold light with the preset brightness range, and outputs the image set shot by the camera module according to the comparison result to obtain image data;

[0117] Initial 3D model module: perform image processing on the image data to obtain a usable image set, extract key nodes from the usable image set, calculate the image distances of all key nodes, obtain the overall image distance set, and obtain the precise 3D coordinates of each key node based on the overall image distance set, and map the precise 3D coordinates to the 3D module to obtain the initial 3D model;

[0118] Final 3D model module: captures key feature values ​​from the available image set and copies the key feature values ​​to the initial 3D model to obtain the final 3D model.

[0119] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A method for obtaining a three-dimensional oral multi-faceted model, characterized in that: The method comprises: Acquire the operating status data of the camera module when shooting the internal image of the oral cavity, and obtain the shooting status coefficient according to the operating status data; the operating status data includes the angle rotation deviation coefficient and the voltage low-frequency coefficient; Compare the shooting state coefficient with a preset shooting state coefficient range, compare the actual brightness of the LED cold light with a preset brightness range, and output a set of images shot by the camera module according to the comparison result to obtain image data; Performing image processing on the image data to obtain a usable image set, extracting key nodes from the usable image set, calculating image distances of all the key nodes to obtain a total image distance set, obtaining precise three-dimensional coordinates of each key node based on the total image distance set, and mapping the precise three-dimensional coordinates to a three-dimensional module to obtain an initial three-dimensional model; Capturing key feature values ​​from the available image set, and copying the key feature values ​​to the initial three-dimensional model to obtain a final three-dimensional model; The shooting state coefficient obtained according to the operating state data includes: The actual rotation angle and the preset rotation angle of the camera module in a fixed time period within a fixed time interval are obtained, and they correspond one to one to obtain a data set. There are n data in the data set, and each data includes an actual rotation angle of the camera module and a corresponding preset rotation angle. The difference between the actual rotation angle and the corresponding preset rotation angle in each data is calculated, and the difference is compared with the preset threshold. If the difference is not less than the preset threshold, the data is marked as 0. If the difference is less than the preset threshold, the data is marked as 1. The number of 0s in the data set is calculated and recorded as m. The angle rotation deviation coefficient is calculated. The calculation expression is: FG=m / n, where FG is the angle rotation deviation coefficient; Obtain the real-time voltage and the preset minimum voltage flowing through the camera module in a fixed time period when the camera module is shooting inside the patient's mouth within a fixed time interval, and calculate the real-time voltage and the preset minimum voltage to obtain the voltage low-frequency coefficient, specifically including: Obtain the real-time voltage and the preset minimum voltage flowing through the camera module in a fixed time period when the camera module shoots inside the patient's mouth within a fixed time interval, and record the time period as h. Compare the real-time voltage flowing through the camera module with the preset minimum voltage. If the real-time voltage flowing through the camera module is less than the preset minimum voltage, record the time period as the acquisition error time period. Calculate the number of acquisition error time periods in all time periods and mark it as i. Calculate the voltage low-frequency coefficient. The calculation formula is: LK=i / h, where LK is the voltage low-frequency coefficient. Calculate the shooting state coefficient, the calculation formula is: Where, Yh is the shooting state coefficient; Comparing the shooting state coefficient with a preset shooting state coefficient range includes: If the shooting state coefficient is within the preset shooting state coefficient range, it is recorded as a normal shooting state; If the shooting state coefficient is not within the preset shooting state coefficient range, it is recorded as abnormal shooting state; Comparison between the actual brightness of the LED cold light and the preset brightness range includes: If the actual brightness of the LED cold light lamp is within the preset brightness range, it is recorded as the brightness of the LED cold light lamp is normal; If the actual brightness of the LED cold light lamp is not within the preset brightness range, it is recorded as abnormal brightness of the LED cold light lamp; When the shooting state is normal and the brightness of the LED cold light lamp is normal, the camera module captures the image set and obtains the image data; otherwise, the camera module and the brightness state of the LED cold light lamp are adjusted, and the image inside the oral cavity is captured again until the shooting state is normal and the brightness of the LED cold light lamp is normal, and the image data is output.

2. The method for obtaining a three-dimensional oral multi-faceted model according to claim 1, characterized in that: The image data is subjected to image processing to obtain a usable image set including: Sorting the image data according to shooting time to obtain a sequence image set; For each pair of consecutive images in the image sequence, the optical flow vector between the two frames is calculated; For each sequence of images, the pixel position is corrected according to the optical flow vector, and at the new pixel position, bilinear interpolation is used to estimate the pixel value at the new position to obtain a usable image set.

3. The method for obtaining a three-dimensional oral multi-faceted model according to claim 1, characterized in that: Extract the key nodes in the available image set, calculate the image distances of all the key nodes, and obtain an image distance set including: Extract all key nodes in the available image set, and calculate the distances of all key nodes in the image containing the key nodes to obtain an image distance set for each key node, and further aggregate the image distance set for each key node to obtain an overall image distance set; And according to the overall image distance set, the precise three-dimensional coordinates of each key node are obtained, and the precise three-dimensional coordinates are mapped to the three-dimensional module to obtain the initial three-dimensional model including: Selecting a reference three-dimensional node in the patient's oral cavity, and assigning the reference three-dimensional node to a preset three-dimensional coordinate, obtaining a basic three-dimensional model, and calculating a three-dimensional distance of the reference three-dimensional node in the basic three-dimensional model; Obtaining a corresponding node in a preset image corresponding to the reference three-dimensional node in the available image set to obtain a marked node, and calculating a two-dimensional image distance of the marked node to obtain a two-dimensional distance of the marked node; Calculating the ratio of the two-dimensional distance of the marked node to the three-dimensional distance of the reference three-dimensional node to obtain a basic ratio; Obtaining the camera module angle corresponding to the basic ratio to obtain an initial angle, extracting the image distance set of each group of key nodes in the image distance set, and obtaining the angle of rotation of the camera module corresponding to different data values ​​in the image distance set of the key node relative to the initial angle to obtain the corresponding actual angle; According to the data value, the basic ratio and the actual angle, the three-dimensional coordinates of the key node corresponding to each data value in the image distance set of the key node are calculated to obtain the three-dimensional coordinate set of the key node; The coordinates in the three-dimensional coordinate set of the key node are averaged to obtain the precise coordinates, and the image distance set corresponding to all the key nodes in the overall image distance set is averaged according to the above steps to obtain the precise coordinates corresponding to each key node, and all the precise coordinates are mapped to the three-dimensional module to obtain the initial three-dimensional model.

4. The method for obtaining a three-dimensional oral multi-faceted model according to claim 1, characterized in that: Capturing key feature values ​​from the available image set and copying the key feature values ​​to the initial three-dimensional model to obtain a final three-dimensional model includes: The contours of tooth crests, troughs and gap points in the available image set are obtained to obtain key feature values, and the key feature values ​​in the available image set are captured and copied to the initial three-dimensional model to obtain the final three-dimensional model.

5. A system for acquiring a three-dimensional oral multi-faceted model, characterized in that: The system includes a camera module, an LED cold light lamp and a back-end analysis device, wherein the LED cold light lamp emits different brightness in real time to supplement the camera module with the image of the inside of the oral cavity, wherein: The operating state data of the camera module when taking images of the interior of the oral cavity, and obtaining a shooting state coefficient according to the operating state data; the operating state data includes an angle rotation deviation coefficient and a voltage low-frequency coefficient; The shooting state coefficient obtained according to the operating state data includes: The actual rotation angle and the preset rotation angle of the camera module in a fixed time period within a fixed time interval are obtained, and they correspond one to one to obtain a data set. There are n data in the data set, and each data includes an actual rotation angle of the camera module and a corresponding preset rotation angle. The difference between the actual rotation angle and the corresponding preset rotation angle in each data is calculated, and the difference is compared with the preset threshold. If the difference is not less than the preset threshold, the data is marked as 0. If the difference is less than the preset threshold, the data is marked as 1. The number of 0s in the data set is calculated and recorded as m. The angle rotation deviation coefficient is calculated. The calculation expression is: FG=m / n, where FG is the angle rotation deviation coefficient; Obtain the real-time voltage and the preset minimum voltage flowing through the camera module in a fixed time period when the camera module is shooting inside the patient's mouth within a fixed time interval, and calculate the real-time voltage and the preset minimum voltage to obtain the voltage low-frequency coefficient, specifically including: Obtain the real-time voltage and the preset minimum voltage flowing through the camera module in a fixed time period when the camera module shoots inside the patient's mouth within a fixed time interval, and record the time period as h. Compare the real-time voltage flowing through the camera module with the preset minimum voltage. If the real-time voltage flowing through the camera module is less than the preset minimum voltage, record the time period as the acquisition error time period. Calculate the number of acquisition error time periods in all time periods and mark it as i. Calculate the voltage low-frequency coefficient. The calculation formula is: LK=i / h, where LK is the voltage low-frequency coefficient. Calculate the shooting state coefficient, the calculation formula is: Where, Yh is the shooting state coefficient; The back-end analysis device compares the shooting state coefficient with a preset shooting state coefficient range, compares the actual brightness of the LED cold light with a preset brightness range, and outputs an image set shot by the camera module according to the comparison result to obtain image data; Comparing the shooting state coefficient with a preset shooting state coefficient range includes: If the shooting state coefficient is within the preset shooting state coefficient range, it is recorded as a normal shooting state; If the shooting state coefficient is not within the preset shooting state coefficient range, it is recorded as abnormal shooting state; Comparison between the actual brightness of the LED cold light and the preset brightness range includes: If the actual brightness of the LED cold light lamp is within the preset brightness range, it is recorded as the brightness of the LED cold light lamp is normal; If the actual brightness of the LED cold light lamp is not within the preset brightness range, it is recorded as abnormal brightness of the LED cold light lamp; When the shooting state is normal and the brightness of the LED cold light lamp is normal, the image set shot by the camera module is output to obtain image data; otherwise, the camera module and the brightness state of the LED cold light lamp are adjusted, and the image inside the oral cavity is shot again until the shooting state is normal and the brightness of the LED cold light lamp is normal, and the image data is output; The back-end analysis device performs image processing on the image data to obtain a usable image set, extracts key nodes from the usable image set, calculates image distances of all key nodes to obtain a total image distance set, obtains precise three-dimensional coordinates of each key node based on the total image distance set, and maps the precise three-dimensional coordinates to a three-dimensional module to obtain an initial three-dimensional model; The back-end analysis device captures key feature values ​​from the available image set, and copies the key feature values ​​to the initial three-dimensional model to obtain a final three-dimensional model.

6. A device for acquiring a three-dimensional oral multi-faceted model, characterized in that: The device comprises: Shooting status module: obtains the operating status data of the camera module when shooting the internal image of the oral cavity, and obtains the shooting status coefficient according to the operating status data; the operating status data includes the angle rotation deviation coefficient and the voltage low-frequency coefficient; The shooting state coefficient obtained according to the operating state data includes: The actual rotation angle and the preset rotation angle of the camera module in a fixed time period within a fixed time interval are obtained, and they correspond one to one to obtain a data set. There are n data in the data set, and each data includes an actual rotation angle of the camera module and a corresponding preset rotation angle. The difference between the actual rotation angle and the corresponding preset rotation angle in each data is calculated, and the difference is compared with the preset threshold. If the difference is not less than the preset threshold, the data is marked as 0. If the difference is less than the preset threshold, the data is marked as 1. The number of 0s in the data set is calculated and recorded as m. The angle rotation deviation coefficient is calculated. The calculation expression is: FG=m / n, where FG is the angle rotation deviation coefficient; Obtain the real-time voltage and the preset minimum voltage flowing through the camera module in a fixed time period when the camera module is shooting inside the patient's mouth within a fixed time interval, and calculate the real-time voltage and the preset minimum voltage to obtain the voltage low-frequency coefficient, specifically including: Obtain the real-time voltage and the preset minimum voltage flowing through the camera module in a fixed time period when the camera module shoots inside the patient's mouth within a fixed time interval, and record the time period as h. Compare the real-time voltage flowing through the camera module with the preset minimum voltage. If the real-time voltage flowing through the camera module is less than the preset minimum voltage, record the time period as the acquisition error time period. Calculate the number of acquisition error time periods in all time periods and mark it as i. Calculate the voltage low-frequency coefficient. The calculation formula is: LK=i / h, where LK is the voltage low-frequency coefficient. Calculate the shooting state coefficient, the calculation formula is: Where, Yh is the shooting state coefficient; Image data module: compares the shooting state coefficient with a preset shooting state coefficient range, compares the actual brightness of the LED cold light with a preset brightness range, and outputs a set of images shot by the camera module according to the comparison result to obtain image data; Comparing the shooting state coefficient with a preset shooting state coefficient range includes: If the shooting state coefficient is within the preset shooting state coefficient range, it is recorded as a normal shooting state; If the shooting state coefficient is not within the preset shooting state coefficient range, it is recorded as abnormal shooting state; Comparison between the actual brightness of the LED cold light and the preset brightness range includes: If the actual brightness of the LED cold light lamp is within the preset brightness range, it is recorded as the brightness of the LED cold light lamp is normal; If the actual brightness of the LED cold light lamp is not within the preset brightness range, it is recorded as abnormal brightness of the LED cold light lamp; When the shooting state is normal and the brightness of the LED cold light lamp is normal, the image set shot by the camera module is output to obtain image data; otherwise, the camera module and the brightness state of the LED cold light lamp are adjusted, and the image inside the oral cavity is shot again until the shooting state is normal and the brightness of the LED cold light lamp is normal, and the image data is output; Initial 3D model module: performing image processing on the image data to obtain a usable image set, extracting key nodes from the usable image set, calculating image distances of all key nodes to obtain a total image distance set, obtaining precise 3D coordinates of each key node based on the total image distance set, and mapping the precise 3D coordinates to the 3D module to obtain an initial 3D model; Final three-dimensional model module: captures key feature values ​​from the available image set, and copies the key feature values ​​to the initial three-dimensional model to obtain a final three-dimensional model.

Images