A model refinement characterization method and system based on image measurement

Abstract

The present application relates to a method and system for fine-tuning a model based on image measurement, the method comprising classifying the image data in response to the acquired image data; using the top surface image, the inner side surface image and the outer side surface image to respectively establish a recognition baseline; determining the mutation area on the top surface recognition baseline, the inner side surface recognition baseline and the outer side surface recognition baseline and matching the mutation area to determine the abnormal position; determining a segmentation line and using the segmentation line to segment the three-dimensional point cloud data to obtain a single tooth model; calculating the vacant area of ​​the single tooth model and using the regenerated three-dimensional point cloud data to fill the vacant area of ​​the single tooth model. The method and system for fine-tuning a model based on image measurement disclosed in the present application segment the three-dimensional point cloud data with the help of images and fuse the repeated data using a position comparison method to obtain an accurate tooth model.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method and system for fine-tuning a model based on image measurement. Background Art

[0002] The refinement of intraoral tooth models is of great significance in the fields of tooth plastic surgery and tooth restoration. First, the methods of obtaining intraoral tooth models are introduced:

[0003] The first method is to obtain the model by solidifying the material. This method is to pour the material into the mold and then perform the bite action. At this time, the material will wrap the teeth. After the material solidifies, take out the material to obtain the model cavity, and then pour the casting material into the model cavity to finally obtain the tooth model. The advantage of this method is that it is easy to operate, but the accuracy of the tooth model is not high because it involves two size conversion processes and is also affected by the accuracy of materials and processes.

[0004] The second method is to use 3D modeling to obtain tooth models, which requires the use of instruments such as intraoral optical scanners. The advantage of this method is that it can directly obtain data with high data accuracy, but further research is needed on tooth classification and repeated data processing.

[0005] Tooth classification refers to grouping data according to teeth, because it is impossible for machines to directly determine the ownership of data. For teeth with a relatively orderly arrangement, it can be determined through simple sorting methods, but for teeth with complex structures, it is difficult to determine the ownership of the data through sorting methods.

[0006] Duplicate data processing refers to the need to fuse data acquired multiple times to fill in missing areas, but the data acquisition benchmark is different each time, so how to fuse them requires further research. Summary of the invention

[0007] The present application provides a method and system for fine-tuning a model based on image measurement, which uses images to segment three-dimensional point cloud data and uses position comparison to fuse repeated data to obtain an accurate tooth model.

[0008] The above-mentioned purpose of the present application is achieved through the following technical solutions:

[0009] In a first aspect, the present application provides a model refinement method based on image measurement, comprising:

[0010] In response to the acquired image data, the image data is classified into top surface images, medial surface images, and lateral surface images;

[0011] Use the top surface image, the inner side surface image and the outer side surface image to respectively establish recognition baselines, thereby obtaining a top surface recognition baseline, an inner side surface recognition baseline and an outer side surface recognition baseline;

[0012] Determine the mutation areas on the top surface recognition baseline, the inner side recognition baseline and the outer side recognition baseline and match the mutation areas to determine the abnormal position;

[0013] Determine a segmentation line according to the top surface identification baseline, the inner side surface identification baseline, the outer side surface identification baseline and the abnormal position, and use the segmentation line to segment the three-dimensional point cloud data to obtain a single tooth model;

[0014] Calculate the missing area of ​​a single tooth model;

[0015] Use the regenerated 3D point cloud data to fill in the missing areas of a single tooth model.

[0016] In a possible implementation of the first aspect, the order of establishing the top surface recognition baseline, the inner side surface recognition baseline, and the outer side surface recognition baseline is to first establish the outer side surface recognition baseline, then establish the top surface recognition baseline, and finally establish the inner side surface recognition baseline.

[0017] In a possible implementation manner of the first aspect, establishing the outer side recognition baseline includes:

[0018] Extracting the contour in the outer side image, the contour includes the edge contour and the bottom contour;

[0019] Determine the bottom contour based on the inner and outer color difference of the contour;

[0020] Identify the arc characteristic line segments included in the bottom contour and determine the dividing points of the arc characteristic line segments;

[0021] The outer side image is segmented once using the dividing point to obtain a plurality of first independent regions, each of which corresponds to a curvature feature line segment;

[0022] extracting a surface included in the first independent region and adjusting the shape of the first independent region so that the first independent region includes only one surface;

[0023] Determine the center point of the segmented surface and sequentially connect the center points of the segmented surface to obtain the outer surface identification baseline;

[0024] The method of establishing the inner side recognition baseline is the same as the method of establishing the outer side recognition baseline.

[0025] In a possible implementation manner of the first aspect, establishing the top surface recognition baseline includes:

[0026] Extracting contour features from the top surface image;

[0027] Use the gradient gap to segment the contour feature once to obtain multiple second independent regions;

[0028] Determine whether to perform secondary segmentation on the second independent region according to the position and area range of the second independent region;

[0029] When the second independent region needs to be segmented twice, the dividing line is determined by using the color difference and the dividing line is used to segment the second independent region twice;

[0030] The center point of the second independent area is determined and the center points of the second independent area are sequentially connected to obtain a top surface identification baseline.

[0031] In a possible implementation manner of the first aspect, determining a mutation region on the top surface recognition baseline, the inner side surface recognition baseline, and the outer side surface recognition baseline, matching the mutation region, and determining the abnormal position includes:

[0032] Transfer the top surface identification baseline, the inner side surface identification baseline and the outer side surface identification baseline into the same coordinate system;

[0033] Match the center points on the top surface identification baseline, the inner side surface identification baseline, and the outer side surface identification baseline to obtain a center point group, each center point group includes three center points;

[0034] When a central point group includes less than three central points, the positions of the central point group are marked as abnormal positions.

[0035] In a possible implementation manner of the first aspect, secondary matching is performed on the central point at the abnormal position, and the secondary matching includes:

[0036] Use the top surface image to determine the medial and lateral exposure areas;

[0037] Determine the center line of the medial exposure area and the center line of the lateral exposure area;

[0038] The center points at the abnormal positions are matched twice by using the center lines of the inner exposed area and the outer exposed area, so that the center point group at the abnormal position includes three center points.

[0039] In a possible implementation of the first aspect, filling a vacant area of ​​a single tooth model using the regenerated three-dimensional point cloud data includes:

[0040] Determine a reference area in the previously generated three-dimensional point cloud data, record it as a pre-reference area, and the number of pre-reference areas is multiple;

[0041] Determining a post-reference region corresponding to the pre-reference region in the regenerated three-dimensional point cloud data;

[0042] Determine the amount of movement when the three-dimensional point cloud data on the front reference area overlaps with the three-dimensional point cloud data on the rear reference area;

[0043] When the movement amount is less than the allowable movement amount, the vacant area of ​​the single tooth model is filled using the regenerated three-dimensional point cloud data.

[0044] In a second aspect, the present application provides a model refinement characterization device based on image measurement, comprising:

[0045] a first image data processing unit, configured to classify the image data in response to the acquired image data, including a top surface image, a medial side surface image, and a lateral side surface image;

[0046] A second image data processing unit is used to establish recognition baselines using the top surface image, the inner side surface image and the outer side surface image, respectively, to obtain a top surface recognition baseline, an inner side surface recognition baseline and an outer side surface recognition baseline;

[0047] A variation region determination unit is used to determine the variation region on the top surface recognition baseline, the inner side surface recognition baseline and the outer side surface recognition baseline and match the variation region to determine the abnormal position;

[0048] A segmentation processing unit, used to determine a segmentation line according to the top surface recognition baseline, the inner side surface recognition baseline, the outer side surface recognition baseline and the abnormal position, and use the segmentation line to segment the three-dimensional point cloud data to obtain a single tooth model;

[0049] A vacancy area calculation unit, used for calculating the vacancy area of ​​a single tooth model;

[0050] The vacant area filling unit is used to fill the vacant area of ​​a single tooth model using the regenerated three-dimensional point cloud data.

[0051] In a third aspect, the present application provides a model refinement characterization system based on image measurement, the system comprising:

[0052] one or more memories for storing instructions; and

[0053] One or more processors, used to call and run the instructions from the memory to execute the method as described in the first aspect and any possible implementation of the first aspect.

[0054] In a fourth aspect, the present application provides a computer-readable storage medium, the computer-readable storage medium comprising:

[0055] Program, when the program is executed by a processor, the method described in the first aspect and any possible implementation of the first aspect is executed.

[0056] In a fifth aspect, the present application provides a computer program product, comprising program instructions. When the program instructions are executed by a computing device, the method described in the first aspect and any possible implementation of the first aspect is executed.

[0057] In a sixth aspect, the present application provides a chip system, which includes a processor for implementing the functions involved in the above aspects, for example, generating, receiving, sending, or processing the data and / or information involved in the above methods.

[0058] The chip system may be composed of chips, or may include chips and other discrete devices.

[0059] In a possible design, the chip system also includes a memory, which is used to store necessary program instructions and data. The processor and the memory can be decoupled and respectively set on different devices, connected by wired or wireless means, or the processor and the memory can also be coupled on the same device. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 This is a schematic flowchart of the steps of a model refinement method based on image measurement provided in this application.

[0061] Figure 2 It is a schematic diagram of an identification baseline provided by this application.

[0062] Figure 3 This is a schematic diagram of unifying three-dimensional point cloud data onto a two-dimensional plane provided by the present application.

[0063] Figure 4 It is a schematic diagram of an edge profile and a bottom profile provided in this application.

[0064] Figure 5 It is a schematic diagram of an arc characteristic line segment provided in this application.

[0065] Figure 6 It is a schematic diagram of a gradual gap provided in this application.

[0066] Figure 7 It is a schematic diagram of an abnormal position provided by this application.

[0067] Figure 8 It is a schematic diagram of an inner exposure area, an outer exposure area and a relative offset angle provided in the present application. DETAILED DESCRIPTION

[0068] The technical solution in this application is further described in detail below in conjunction with the accompanying drawings.

[0069] This application discloses a model refinement method based on image measurement, please refer to Figure 1 In some examples, the model refinement method based on image measurement disclosed in this application includes the following steps:

[0070] S101, in response to the acquired image data, classifying the image data into top surface images, medial surface images and lateral surface images;

[0071] S102, using the top surface image, the inner side surface image and the outer side surface image to respectively establish recognition baselines, thereby obtaining a top surface recognition baseline, an inner side surface recognition baseline and an outer side surface recognition baseline;

[0072] S103, determining the mutation area on the top surface recognition baseline, the inner side surface recognition baseline and the outer side surface recognition baseline and matching the mutation area to determine the abnormal position;

[0073] S104, determining a segmentation line according to the top surface recognition baseline, the inner side surface recognition baseline, the outer side surface recognition baseline and the abnormal position, and using the segmentation line to segment the three-dimensional point cloud data to obtain a single tooth model;

[0074] S105, calculating the vacant area of ​​a single tooth model;

[0075] S106, using the regenerated three-dimensional point cloud data to fill in the vacant area of ​​the single tooth model.

[0076] First of all, it needs to be explained that the model refinement characterization method based on image measurement disclosed in this application is applied to a processor, which acquires data through a scanning device (equipped with an image sensor). The acquired data is divided into two categories, namely image data and three-dimensional point cloud data.

[0077] The advantage of image data is that it is intuitive, but its ability to respond to local details and depth information is poor. The advantage of 3D point cloud data is data accuracy, but it is difficult to segment and process, and it requires multiple data superpositions to ensure the data integrity of the coverage area. In addition, 3D point cloud data is a kind of range data, which needs to be segmented to obtain the 3D point cloud data belonging to the teeth from the range data.

[0078] In step S101, the received image data is first classified into three categories, namely, top surface image, medial side surface image and lateral side surface image. Then, in step S102, the top surface image, medial side surface image and lateral side surface image are used to establish recognition baselines respectively, and the top surface recognition baseline, the medial side surface recognition baseline and the lateral side surface recognition baseline are obtained. Figure 2 shown.

[0079] Then in step S103, the mutation areas on the top surface recognition baseline, the inner side surface recognition baseline and the outer side surface recognition baseline are determined and matched to determine the abnormal position. The abnormal position here refers to the area where the teeth are not arranged neatly, and the main reasons here are overlap and missing teeth.

[0080] In step S104, the top surface identification baseline, the inner side surface identification baseline, the outer side surface identification baseline and the abnormal position are used to determine the segmentation line, and the segmentation line is used to segment the three-dimensional point cloud data to obtain a single tooth model.

[0081] At this time, there will be a certain amount of vacant areas on the single tooth model. The vacant areas are determined in step S105, and then in step S106, the vacant areas of the single tooth model are filled using the regenerated three-dimensional point cloud data.

[0082] It should be understood that three-dimensional point cloud data expresses objects through spatial data points. When the number of spatial data points is small, there will be a lack of expression accuracy. There is also the problem of unclear boundaries, especially for the boundary area of ​​teeth.

[0083] With the help of images, teeth can be quickly segmented, and after the teeth are segmented, their boundaries can be quickly identified, thereby realizing rapid screening of three-dimensional point cloud data. At this time, multiple sets of three-dimensional point cloud data can be superimposed to obtain a tooth model with higher accuracy.

[0084] The image is displayed in two-dimensional space, and the three-dimensional point cloud data is displayed in three-dimensional space. At this time, it is necessary to unify them into a two-dimensional plane, such as Figure 3 As shown, the three-dimensional point cloud data needs to be projected on a plane, and here it is limited to use in the process of segmenting the three-dimensional point cloud data using the segmentation line.

[0085] In some examples, the order of establishing the top surface identification baseline, the inner side surface identification baseline and the outer side surface identification baseline is to first establish the outer side surface identification baseline, then establish the top surface identification baseline, and finally establish the inner side surface identification baseline.

[0086] The specific method of establishing the outer side identification baseline is as follows:

[0087] S201, extracting the contour in the outer surface image, the contour including the edge contour and the bottom contour;

[0088] S202, determining the bottom contour according to the inner and outer color difference of the contour;

[0089] S203, identifying the curvature characteristic line segments included in the bottom contour and determining the dividing points of the curvature characteristic line segments;

[0090] S204, using the demarcation point to segment the outer side image once, to obtain a plurality of first independent regions, each of which corresponds to a curvature feature line segment;

[0091] S205, extracting a surface included in the first independent region and adjusting the shape of the first independent region so that the first independent region includes only one surface;

[0092] S206, determining the center point of the segmented surface and sequentially connecting the center points of the segmented surface to obtain an outer surface identification baseline;

[0093] The method of establishing the inner side recognition baseline is the same as the method of establishing the outer side recognition baseline.

[0094] The process of the above method is to first obtain the contour in the outer surface image, where the contour refers to the contour of the tooth, because the color of the tooth is obviously different from the color of other areas.

[0095] See also Figure 4 The contour is divided into two parts, namely the edge contour and the bottom contour. The bottom contour refers to the contour on the tooth that contacts the gum part, and the edge contour refers to the contour other than the bottom contour.

[0096] From the above content, we can know that the bottom contour can be directly determined using the inner and outer color difference of the contour, and then the arc feature line segments included in the bottom contour are identified and the dividing point of the arc feature line segments is determined. The dividing point refers to the dividing point of two teeth.

[0097] The number of dividing points between two teeth may be one or two, and the number of dividing points mainly depends on the distance between the roots of the two teeth. When the distance between the roots of the two teeth is long, tooth loss and root resorption may occur.

[0098] In step S204, the outer side image is segmented once using the dividing point to obtain a plurality of first independent regions, each of which corresponds to a curvature feature line segment. Figure 5 As shown, at this time, a straight line is generally established at the dividing point, and the area between two adjacent straight lines is called the first independent area.

[0099] Then in step S205, the surface included in the first independent region is extracted and the shape of the first independent region is adjusted so that the first independent region includes only one surface, that is, each first independent region corresponds to the surface of only one tooth.

[0100] For a tooth, once its root is determined, the tooth contour can be determined based on the point on the tooth root. Finally, in step S206, the center point of the segmented surface is determined and the center points of the segmented surface are sequentially connected to obtain the outer surface recognition baseline.

[0101] The method of establishing the inner side identification baseline is the same as that of establishing the outer side identification baseline, which will not be repeated here.

[0102] The center point on the outer side surface recognition baseline is also called the center point of the outer side surface recognition baseline, which will be used in the subsequent matching process.

[0103] The top surface identification baseline is established as follows:

[0104] S301, extracting contour features in the top surface image;

[0105] S302, using a gradient gap to segment the contour feature once to obtain a plurality of second independent regions;

[0106] S303, determining whether to perform secondary segmentation on the second independent region according to the position and area range of the second independent region;

[0107] S304, when the second independent region needs to be segmented twice, a dividing line is determined by using the color difference and the dividing line is used to segment the second independent region twice;

[0108] S305, determining the center point of the second independent area and sequentially connecting the center points of the second independent area to obtain a top surface identification baseline.

[0109] In the above steps, it is necessary to use a gradient gap to segment the contour feature once to obtain multiple second independent regions. Figure 6 The gradient gap here refers to the shape of the gum tissue in the area around the teeth. Because there are arc chamfers in the surrounding shape of the teeth, there will be gradient gaps between adjacent teeth.

[0110] The width of the tapered gap tends to decrease as it approaches the center of the tooth.

[0111] After the second independent region is obtained, whether to perform secondary segmentation on the second independent region will be determined according to the position and area range of the second independent region. When the second independent region needs to be segmented secondary, the dividing line is determined by color difference and the dividing line is used to perform secondary segmentation on the second independent region.

[0112] Here, using color difference to determine the dividing line refers to using the edge of the tooth to form the dividing line, and then performing a secondary segmentation on the second independent area.

[0113] Finally, the center point of the second independent area is determined and the center points of the second independent area are sequentially connected to obtain the top surface identification baseline.

[0114] So far we have obtained the top surface identification baseline, the inner side identification baseline and the outer side identification baseline. If the position of each center point on the top surface identification baseline, the inner side identification baseline and the outer side identification baseline is accurate, then for a tooth, there will be a center point on the top surface identification baseline, the inner side identification baseline and the outer side identification baseline respectively.

[0115] In some examples, the method of determining the mutation area on the top surface identification baseline, the inner side surface identification baseline, and the outer side surface identification baseline, matching the mutation area, and determining the abnormal position is as follows:

[0116] Transfer the top surface identification baseline, the inner side surface identification baseline and the outer side surface identification baseline into the same coordinate system;

[0117] Match the center points on the top surface identification baseline, the inner side surface identification baseline, and the outer side surface identification baseline to obtain a center point group, each center point group includes three center points;

[0118] When a central point group includes less than three central points, the positions of the central point group are marked as abnormal positions.

[0119] A center point group includes three center points, which means that a tooth has a center point on the top surface identification baseline, the inner side surface identification baseline and the outer side surface identification baseline respectively, and the positional relationship of the three center points is relatively fixed.

[0120] One way to determine this is to proportionally reduce and move the outer side identification baseline, whereupon the center point on the outer side identification baseline should coincide with the corresponding center point on the top surface identification baseline and the corresponding center point on the inner side identification baseline in sequence.

[0121] When this method cannot be achieved, it means that there is an abnormality in the tooth posture or tooth relationship, such as Figure 7 As shown in the dotted area, further confirmation is needed. One reason for the abnormality is that the teeth overlap or tilt, leading to misjudgment.

[0122] At this time, the area that cannot be determined is called an abnormal position, and the center point of the abnormal position needs to be matched twice. The method of secondary matching is as follows:

[0123] Use the top surface image to determine the medial and lateral exposure areas;

[0124] Determine the center line of the medial exposure area and the center line of the lateral exposure area;

[0125] The center points at the abnormal positions are matched twice by using the center lines of the inner exposed area and the outer exposed area, so that the center point group at the abnormal position includes three center points.

[0126] The inner exposure area and the outer exposure area are determined by an occlusion relationship.

[0127] See also Figure 8 The core of the secondary matching is to determine the position of the center point through the inner exposure area and the outer exposure area of ​​the top surface image, and then use the center line of the inner exposure area and the center line of the outer exposure area to perform secondary matching on the center point at the abnormal position, so that the center point group at the abnormal position includes three center points.

[0128] In some possible implementations, the method of performing secondary matching on the center point at the abnormal position with the help of the center line of the inner exposure area and the center line of the outer exposure area is to determine the relative offset angle of the center line corresponding to the center point based on the relative position of the center line and the center point of the tooth on the top surface image, and then use the relative offset angle to correct the position of the center line corresponding to the center point.

[0129] It should be noted here that the relative offset angle first determines the direction. At this time, the angle between the two center points involved in the correction should be smaller than the relative offset angle.

[0130] At this time, it should be noted that for the inner and outer side surface identification baselines or the center point on the outer side surface identification baseline, the direction of the center point also needs to be determined so as to be associated with the center point on the top surface identification baseline.

[0131] In some possible implementations, the direction of the center point is the direction of the midline of the surface where the center point is located.

[0132] In some examples, the method of filling the missing areas of a single tooth model using the regenerated 3D point cloud data is:

[0133] S401, determining a reference area in the three-dimensional point cloud data generated last time, recorded as a pre-reference area, and the number of pre-reference areas is multiple;

[0134] S402, determining a post-reference area corresponding to the pre-reference area in the regenerated three-dimensional point cloud data;

[0135] S403, determining a movement amount when the three-dimensional point cloud data on the front reference area overlaps with the three-dimensional point cloud data on the rear reference area;

[0136] S404: When the movement amount is less than the allowable movement amount, the vacant area of ​​the single tooth model is filled using the regenerated three-dimensional point cloud data.

[0137] The contents of step S401 to step S404 use the reference area in the twice generated three-dimensional point cloud data to determine the movement amount, and then the movement amount when the three-dimensional point cloud data on the front reference area overlaps with the three-dimensional point cloud data on the rear reference area.

[0138] When the movement amount is less than the allowable movement amount, it indicates that the regenerated three-dimensional point cloud data meets the use requirements, and the regenerated three-dimensional point cloud data can be used to fill the vacant area of ​​the single tooth model.

[0139] When the movement amount is greater than or equal to the allowed movement amount, it means that the device is moving too fast or a certain amount of data is lost, and the 3D point cloud data needs to be re-acquired and regenerated.

[0140] The present application also provides a model refinement characterization device based on image measurement, comprising:

[0141] a first image data processing unit, configured to classify the image data in response to the acquired image data, including a top surface image, a medial side surface image, and a lateral side surface image;

[0142] A second image data processing unit is used to establish recognition baselines using the top surface image, the inner side surface image and the outer side surface image, respectively, to obtain a top surface recognition baseline, an inner side surface recognition baseline and an outer side surface recognition baseline;

[0143] A variation region determination unit is used to determine the variation region on the top surface recognition baseline, the inner side surface recognition baseline and the outer side surface recognition baseline and match the variation region to determine the abnormal position;

[0144] A segmentation processing unit, used to determine a segmentation line according to the top surface recognition baseline, the inner side surface recognition baseline, the outer side surface recognition baseline and the abnormal position, and use the segmentation line to segment the three-dimensional point cloud data to obtain a single tooth model;

[0145] A vacancy area calculation unit, used for calculating the vacancy area of ​​a single tooth model;

[0146] The vacant area filling unit is used to fill the vacant area of ​​a single tooth model using the regenerated three-dimensional point cloud data.

[0147] Furthermore, the order of establishing the top surface identification baseline, the inner side surface identification baseline and the outer side surface identification baseline is to first establish the outer side surface identification baseline, then establish the top surface identification baseline, and finally establish the inner side surface identification baseline.

[0148] Furthermore, it also includes:

[0149] A contour extraction unit, used to extract the contour in the outer side image, the contour including the edge contour and the bottom contour;

[0150] A contour determination unit, used for determining a bottom contour according to the inner and outer color differences of the contour;

[0151] An identification unit, used for identifying the arc characteristic line segments included in the bottom contour and determining the dividing points of the arc characteristic line segments;

[0152] A primary segmentation unit, used for performing a primary segmentation on the outer side image using a dividing point to obtain a plurality of first independent regions, each of which corresponds to a curvature feature line segment;

[0153] A first extraction unit, used to extract a surface included in the first independent region and adjust the shape of the first independent region so that the first independent region includes only one surface;

[0154] A center point determination unit, used to determine the center point of the segmented surface and sequentially connect the center points of the segmented surface to obtain an outer side surface identification baseline;

[0155] The method of establishing the inner side recognition baseline is the same as the method of establishing the outer side recognition baseline.

[0156] Furthermore, it also includes:

[0157] A second extraction unit is used to extract contour features in the top surface image;

[0158] A first segmentation processing unit is used to segment the contour feature once using a gradient gap to obtain a plurality of second independent regions;

[0159] A first determining unit, configured to determine whether to perform secondary segmentation on the second independent region according to the position and area range of the second independent region;

[0160] When the second independent region needs to be segmented twice, the dividing line is determined by using the color difference and the dividing line is used to segment the second independent region twice;

[0161] The center point of the second independent area is determined and the center points of the second independent area are sequentially connected to obtain a top surface identification baseline.

[0162] Furthermore, it also includes:

[0163] A first processing unit is used to transfer the top surface identification baseline, the inner side surface identification baseline and the outer side surface identification baseline into the same coordinate system;

[0164] A matching processing unit, used for matching the center points on the top surface identification baseline, the inner side surface identification baseline and the outer side surface identification baseline to obtain a center point group, each center point group includes three center points;

[0165] The abnormal position marking unit is used to mark the position of the central point group as an abnormal position when a central point group includes less than three central points.

[0166] Furthermore, it also includes:

[0167] a second determining unit, configured to determine an inner exposure area and an outer exposure area using the top surface image;

[0168] A third determining unit, used to determine a center line of the inner exposure area and a center line of the outer exposure area;

[0169] The secondary matching unit is used to perform secondary matching on the center point at the abnormal position by means of the center line of the inner exposure area and the center line of the outer exposure area, so that the center point group at the abnormal position includes three center points.

[0170] Furthermore, it also includes:

[0171] A reference region determination unit is used to determine a reference region in the three-dimensional point cloud data generated last time, which is recorded as a pre-reference region. The number of pre-reference regions is multiple;

[0172] A post-reference region determining unit, used to determine a post-reference region corresponding to the pre-reference region in the regenerated three-dimensional point cloud data;

[0173] A movement amount determination unit, used to determine the movement amount when the three-dimensional point cloud data on the front reference area overlaps with the three-dimensional point cloud data on the rear reference area;

[0174] The filling unit is used to fill the vacant area of ​​a single tooth model using regenerated three-dimensional point cloud data when the movement amount is less than the allowed movement amount.

[0175] In one example, the unit in any of the above devices can be one or more integrated circuits configured to implement the above methods, such as: one or more application specific integrated circuits (ASICs), or, one or more digital signal processors (DSPs), or, one or more field programmable gate arrays (FPGAs), or a combination of at least two of these integrated circuit forms.

[0176] For another example, when the units in the device can be implemented in the form of a processing element scheduling program, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call a program. For another example, these units can be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0177] Various objects such as various messages / information / equipment / network elements / systems / devices / actions / operations / processes / concepts that may appear in this application are named. It can be understood that these specific names do not constitute a limitation on the relevant objects. The names assigned may change with factors such as scenarios, contexts or usage habits. The understanding of the technical meaning of the technical terms in this application should be mainly determined from the functions and technical effects embodied / executed in the technical scheme.

[0178] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0179] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0180] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0181] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0182] It should also be understood that in various embodiments of the present application, the first, second, etc. are only used to indicate that multiple objects are different. For example, the first time window and the second time window are only used to indicate different time windows. They should not have any impact on the time window itself, and the first, second, etc. mentioned above should not impose any limitations on the embodiments of the present application.

[0183] It should also be understood that in the various embodiments of the present application, unless otherwise specified or there is a logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referenced to each other, and the technical features in different embodiments can be combined to form new embodiments according to their internal logical relationships.

[0184] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a computer-readable storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned computer-readable storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0185] The present application also provides a model refinement characterization system based on image measurement, the system comprising:

[0186] one or more memories for storing instructions; and

[0187] One or more processors are used to call and run the instructions from the memory to execute the method as described above.

[0188] The present application also provides a computer program product, which includes instructions. When the instructions are executed, the terminal device and the network device perform operations of the terminal device and the network device corresponding to the above method.

[0189] The present application also provides a chip system, which includes a processor for implementing the functions involved in the above content, such as generating, receiving, sending, or processing the data and / or information involved in the above method.

[0190] The chip system may be composed of chips, or may include chips and other discrete devices.

[0191] The processor mentioned in any of the above places can be a CPU, a microprocessor, an ASIC, or one or more integrated circuits for executing programs for controlling the above-mentioned feedback information transmission method.

[0192] In a possible design, the chip system also includes a memory, which is used to store necessary program instructions and data. The processor and the memory can be decoupled and respectively set on different devices, connected by wire or wireless means to support the chip system to implement various functions in the above embodiments. Alternatively, the processor and the memory can also be coupled on the same device.

[0193] Optionally, the computer instructions are stored in a memory.

[0194] Optionally, the memory is a storage unit within the chip, such as a register, a cache, etc. The memory can also be a storage unit within the terminal located outside the chip, such as a ROM or other types of static storage devices that can store static information and instructions, RAM, etc.

[0195] It can be understood that the memory in the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories.

[0196] The nonvolatile memory may be a ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), or a flash memory.

[0197] The volatile memory may be a RAM, which is used as an external cache. There are many different types of RAM, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory.

[0198] The embodiments of this specific implementation method are all preferred embodiments of the present application, and are not intended to limit the protection scope of the present application. Therefore, all equivalent changes made based on the structure, shape, and principle of the present application should be included in the protection scope of the present application.

Claims

1. A model refinement method based on image measurement, characterized in that: include: In response to the acquired image data, the image data is classified into top surface images, medial surface images, and lateral surface images; Use the top surface image, the inner side surface image and the outer side surface image to respectively establish recognition baselines, thereby obtaining a top surface recognition baseline, an inner side surface recognition baseline and an outer side surface recognition baseline; Determine the mutation areas on the top surface recognition baseline, the inner side recognition baseline and the outer side recognition baseline and match the mutation areas to determine the abnormal position; Determine a segmentation line according to the top surface identification baseline, the inner side surface identification baseline, the outer side surface identification baseline and the abnormal position, and use the segmentation line to segment the three-dimensional point cloud data to obtain a single tooth model; Calculate the missing area of ​​a single tooth model; Use the regenerated 3D point cloud data to fill in the missing areas of a single tooth model; Determining the mutation area on the top surface identification baseline, the inner side identification baseline and the outer side identification baseline, matching the mutation area and determining the abnormal position includes: Transfer the top surface identification baseline, the inner side surface identification baseline and the outer side surface identification baseline into the same coordinate system; Match the center points on the top surface identification baseline, the inner side surface identification baseline, and the outer side surface identification baseline to obtain a center point group, each center point group includes three center points; When a center point group includes less than three center points, marking the position of the center point group as an abnormal position; Establishing a baseline for external face identification includes: Extracting the contour in the outer side image, the contour includes the edge contour and the bottom contour; Determine the bottom contour based on the inner and outer color difference of the contour; Identify the arc characteristic line segments included in the bottom contour and determine the dividing points of the arc characteristic line segments; The outer side image is segmented once using the dividing point to obtain a plurality of first independent regions, each of which corresponds to a curvature feature line segment; extracting a surface included in the first independent region and adjusting the shape of the first independent region so that the first independent region includes only one surface; Determine the center point of the segmented surface and sequentially connect the center points of the segmented surface to obtain the outer surface identification baseline; The method of establishing the inner side identification baseline is the same as the method of establishing the outer side identification baseline; Establishing a top surface identification baseline includes: Extracting contour features from the top surface image; Use the gradient gap to segment the contour feature once to obtain multiple second independent regions; Determine whether to perform secondary segmentation on the second independent region according to the position and area range of the second independent region; When the second independent region needs to be segmented twice, the dividing line is determined by using the color difference and the dividing line is used to segment the second independent region twice; The center point of the second independent area is determined and the center points of the second independent area are sequentially connected to obtain a top surface identification baseline.

2. The model refinement characterization method based on image measurement according to claim 1 is characterized in that: The order of establishing the top surface identification baseline, the inner side surface identification baseline and the outer side surface identification baseline is to first establish the outer side surface identification baseline, then establish the top surface identification baseline, and finally establish the inner side surface identification baseline.

3. The model refinement characterization method based on image measurement according to claim 1 is characterized in that: Perform secondary matching on the center point at the abnormal position. The secondary matching includes: Use the top surface image to determine the medial and lateral exposure areas; Determine the center line of the medial exposure area and the center line of the lateral exposure area; The center points at the abnormal positions are matched twice by using the center lines of the inner exposed area and the outer exposed area, so that the center point group at the abnormal position includes three center points.

4. The model refinement characterization method based on image measurement according to claim 1 is characterized in that: Filling in the gaps in a single tooth model using regenerated 3D point cloud data includes: Determine a reference area in the previously generated three-dimensional point cloud data, record it as a pre-reference area, and the number of pre-reference areas is multiple; Determining a rear reference area corresponding to the front reference area in the regenerated three-dimensional point cloud data; Determine the amount of movement when the three-dimensional point cloud data on the front reference area overlaps with the three-dimensional point cloud data on the rear reference area; When the movement amount is less than the allowable movement amount, the vacant area of ​​the single tooth model is filled using the regenerated three-dimensional point cloud data.

5. A model refinement characterization system based on image measurement, characterized in that: The system comprises: one or more memories for storing instructions; and One or more processors, configured to call and execute the instructions from the memory to perform the method according to any one of claims 1 to 4.

Images