Model generation method and apparatus, and electronic device and storage medium

Abstract

Provided in the present application are a model generation method and apparatus, and an electronic device and a storage medium. The method comprises: acquiring a first three-dimensional model and a second three-dimensional model of a target object; determining a region of interest in the second three-dimensional model; expanding the region of interest to determine a region to be stitched in the second three-dimensional model; stitching the first three-dimensional model with the region to be stitched in the second three-dimensional model, so as to obtain stitching data, and on the basis of the stitching data, determining whether the first three-dimensional model matches the second three-dimensional model; and when it is determined that the first three-dimensional model matches the second three-dimensional model, fusing the first three-dimensional model with the second three-dimensional model on the basis of the stitching data, so as to obtain a target three-dimensional model. In the present application, a complete target three-dimensional model that reflects a true shape of a target object can be rapidly generated on the basis of a first three-dimensional model and a second three-dimensional model, and the target three-dimensional model has relatively high accuracy, thereby achieving a good presentation effect of the true shape of a region of interest of the target object.

Description

Model generation methods, apparatus, electronic devices and storage media

[0001] This disclosure claims priority to Chinese Patent Application No. 202411638939.0, filed on November 15, 2024, entitled “Model Generation Method, Electronic Device and Storage Medium”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of scanning processing technology, and in particular to a model generation method, apparatus, electronic device and storage medium. Background Technology

[0003] 3D scanning is widely used in product design and manufacturing. For example, scanning a first 3D model of an object can be used for dental restoration design or industrial product design. However, the first 3D model cannot accurately represent all areas of the object, or scanning the first 3D model may result in unclear images of certain areas, making it impossible to generate a complete 3D model that accurately reflects the object's shape.

[0004] For example, dental restoration design includes post-core crowns, inlays, full crowns, prostheses, occlusal plates, and dental implants. In root canal treatment, the post-core plaster cast and impression can be scanned, and the post-core crown can be fabricated based on the scanned data. Root canal treatment is a common method for treating pulpitis and periapical periodontitis. In the root canal, the "load-bearing pillar" of the entire restoration is called the "post." Fixed to the post, together with the remaining tooth structure, forms a preparation for full crown retention, called the core. The post not only provides retention but also transmits external forces from the crown, core, and remaining crown structure. Typically, adding a post and core provides support and retention for the full crown, upon which a full crown restoration can be performed. However, due to the narrowness of the post channel, it is difficult to clearly scan the shape of the post channel.

[0005] Similarly, for some industrial products, there is also the problem of difficulty in scanning certain channels. Summary of the Invention

[0006] This application provides a model generation method, apparatus, electronic device, and storage medium to quickly generate a complete target 3D model that reflects the true shape of a target object based on a first 3D model and a second 3D model. The target 3D model has high accuracy and good representation of the true shape of the region of interest of the target object.

[0007] In a first aspect, embodiments of this application provide a model generation method, the method comprising: acquiring a first three-dimensional model and a second three-dimensional model of a target object; determining a region of interest in the second three-dimensional model; expanding the region of interest to determine a region to be stitched in the second three-dimensional model; stitching the first three-dimensional model and the region to be stitched in the second three-dimensional model to obtain stitching data, and determining whether the first three-dimensional model and the second three-dimensional model match based on the stitching data; when it is determined that the first three-dimensional model and the second three-dimensional model match, fusing the first three-dimensional model and the second three-dimensional model based on the stitching data to obtain a target three-dimensional model.

[0008] In some embodiments of this application, fusing the first three-dimensional model and the second three-dimensional model according to the stitching data to obtain a target three-dimensional model includes: deleting the portion of the region to be stitched from the stitching data other than the region of interest; performing a de-overlapping operation on the stitching data to obtain an initial fused model; and performing a de-noising operation on the initial fused model to obtain the target three-dimensional model.

[0009] In some embodiments of this application, the step of performing a de-overlapping operation on the stitched data to obtain an initial fusion model includes: assigning correlation values ​​to the three-dimensional points in the stitched data, wherein the correlation values ​​represent the correlation between the three-dimensional points and the initial fusion model; performing a de-overlapping operation on the three-dimensional points in the stitched data based on the correlation values ​​of the three-dimensional points to obtain the initial fusion model; wherein the correlation values ​​of the three-dimensional points in the region of interest in the second three-dimensional model are the highest, and the correlation values ​​of the three-dimensional points in the first three-dimensional model are greater than the correlation values ​​of the three-dimensional points in the second three-dimensional model.

[0010] In some embodiments of this application, the denoising operation on the initial fusion model includes: deleting target data blocks in the initial fusion model, wherein the target data blocks include fewer than a preset number of three-dimensional points.

[0011] In some embodiments of this application, determining the region of interest in the second three-dimensional model includes: determining the marking mode selected by the user; if the marking mode is a first mode, marking the three-dimensional points in the second three-dimensional model where the user performs operations using a preset graphic; if the marking mode is a second mode, determining the range of the operation area in the second three-dimensional model where the user performs operations, and marking the three-dimensional points within the operation area; and taking the marked area formed by the marked three-dimensional points as the region of interest.

[0012] In some embodiments of this application, determining the region of interest in the second three-dimensional model includes: determining the region of interest in the second three-dimensional model using a preset AI model.

[0013] In some embodiments of this application, expanding the region of interest and determining the region to be stitched in the second three-dimensional model includes: extracting the bounding box of the region of interest, expanding the bounding box outward according to a preset range to obtain an expanded bounding box; and determining the region to be stitched based on the expanded bounding box.

[0014] In some embodiments of this application, before splicing the first three-dimensional model with the area to be spliced ​​in the second three-dimensional model, the method further includes: flipping the second three-dimensional model along a preset direction so that the area to be spliced ​​in the second three-dimensional model is flipped, wherein the preset direction is the normal of the area to be spliced.

[0015] In some embodiments of this application, when it is determined that the first 3D model and the second 3D model do not match, the method further includes: displaying the area to be stitched and issuing a first prompt message to enable the user to perform a point selection operation within the area to be stitched; if the user is detected performing a point selection operation within the area to be stitched, responding to the point selection operation and determining multiple stitching points; stitching the first 3D model and the area to be stitched based on the multiple stitching points; if the user is detected not performing a point selection operation within the area to be stitched, issuing a feedback message to enable the user to perform a point selection operation within the area to be stitched.

[0016] In some embodiments of this application, determining whether the first 3D model and the second 3D model match based on the stitching data includes: determining a comparison region in the first 3D model corresponding to the region to be stitched; calculating a matching value between the comparison region and the region to be stitched, wherein the matching value represents the degree of matching between the comparison region and the region to be stitched; if the matching value is less than a preset threshold, determining that the first 3D model and the second 3D model do not match; if the matching value is greater than or equal to the preset threshold, determining that the first 3D model and the second 3D model match; or, determining whether the first 3D model and the second 3D model match based on the stitching data includes: issuing a second prompt message to prompt the user to confirm whether the first 3D model and the second 3D model match; if the user's confirmation operation is received, determining that the first 3D model and the second 3D model match; if the user's confirmation operation is not received, determining that the first 3D model and the second 3D model do not match.

[0017] Secondly, embodiments of this application provide an electronic device, including a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the electronic device executes the above-described model generation method.

[0018] Thirdly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-described model generation method.

[0019] Fourthly, embodiments of this application provide a model generation apparatus, wherein the model generation apparatus includes: an acquisition module, configured to acquire a first three-dimensional model and a second three-dimensional model of a target object; a first determination module, configured to determine a region of interest in the second three-dimensional model; a second determination module, configured to expand the region of interest and determine a region to be stitched in the second three-dimensional model; a first stitching module, configured to stitch the first three-dimensional model and the region to be stitched in the second three-dimensional model to obtain stitching data, and determine whether the first three-dimensional model and the second three-dimensional model match based on the stitching data; and a fusion module, configured to, when it is determined that the first three-dimensional model and the second three-dimensional model match, fuse the first three-dimensional model and the second three-dimensional model based on the stitching data to obtain a target three-dimensional model.

[0020] This application expands the region of interest (ROI) of the second 3D model to determine the region to be stitched within the second 3D model. It then stitches the first 3D model with the region to be stitched in the second 3D model to obtain stitched data. When the two models are determined to match based on the stitched data, they are fused to obtain a complete target 3D model. This allows for the rapid generation of a complete target 3D model reflecting the true shape of the target object based on the first and second 3D models. Furthermore, this target 3D model has high accuracy and good representation of the ROI, avoiding the problems of existing target 3D models failing to accurately reflect the ROI or having unclear scans of the first 3D model, which prevent the generation of a complete 3D model reflecting the true shape of the object. In addition, by expanding the ROI to obtain the region to be stitched and stitching the first and second 3D models based on this region, this application ensures sufficient common area between the two models for stitching, thereby improving stitching accuracy. This application also removes the portion of the region to be stitched from the stitching data between the first 3D model and the second 3D model, excluding the region of interest, and performs a de-overlapping point operation on the stitching data to obtain an initial fused model. This can effectively reduce the overlapping data of the initial fused model of the two models. Thus, there is no need to edit or delete the overlapping data in the first 3D model and the second 3D model, which simplifies the generation process of the target 3D model, improves the design efficiency of the target 3D model, and reduces the error rate of the user's deletion operation. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 is a flowchart illustrating the model generation method provided in some embodiments of this application.

[0023] Figure 2 is a schematic diagram of a first three-dimensional model provided in some embodiments of this application.

[0024] Figure 3 is a schematic diagram of a second three-dimensional model provided in some embodiments of this application.

[0025] Figure 4 is a schematic diagram of the first interface provided in some embodiments of this application.

[0026] Figure 5 is a flowchart of a method for matching a first three-dimensional model and a second three-dimensional model provided in some embodiments of this application.

[0027] Figure 6 is a flowchart of a method for matching a first three-dimensional model and a second three-dimensional model provided in some other embodiments of this application.

[0028] Figure 7 is a schematic diagram of a second interface provided in some embodiments of this application.

[0029] Figure 8 is a flowchart of a method for fusing a first three-dimensional model and a second three-dimensional model in some embodiments of this application.

[0030] Figure 9 is a schematic diagram of the target three-dimensional model in some embodiments of this application.

[0031] Figure 10 is a structural diagram of a model generation apparatus provided in some embodiments of this application.

[0032] Figure 11 is a schematic diagram of an electronic device provided in some embodiments of this application. Detailed Implementation

[0033] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0034] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0035] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this application's specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. It should be understood that, unless otherwise stated, "at least one" means one or more. "More than one" means two or more. For example, at least one of a, b, or c can represent seven cases: a, b, c, a and b, a and c, b and c, and a, b, and c.

[0036] The functional units and modules in the above embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of the embodiments of this application.

[0037] 3D scanning is widely used in product design and manufacturing. For example, scanning 3D plaster models of objects can be used for dental restoration design or industrial product design. However, 3D plaster models cannot accurately represent all areas of an object, or scanning 3D plaster models may result in unclear scans of certain areas, making it impossible to generate a complete 3D model that accurately reflects the object's shape.

[0038] One solution is to stitch and merge the 3D plaster model and the 3D impression model to obtain data for the region of interest. However, the process of stitching and merging the 3D plaster model and the 3D impression model is complex, and the merged model contains a lot of miscellaneous data, leading to a high failure rate and ultimately preventing the generation of a complete target 3D model. For example, in related technologies, to obtain a complete target 3D model, after obtaining the 3D plaster model and the 3D impression model through 3D scanning, the scanned models need to be edited to delete useless miscellaneous data (such as data related to holes in the model). Then, the data from the 3D plaster model and the 3D impression model are stitched and merged to obtain the complete target 3D model. However, the deletion range needs to be controlled when deleting the scanned data. If the deletion range is too large, too much data may be deleted, reducing the common area between the 3D plaster model and the 3D impression model. A smaller common area makes stitching the 3D plaster model and the 3D impression model difficult and results in a high failure rate. If the deletion range is too small, it is easy to cause too much miscellaneous data after merging the 3D plaster model and the 3D impression model, resulting in merging failure and thus failing to generate a complete target 3D model for dental restoration design or industrial product design.

[0039] This application provides a model generation method, electronic device, and storage medium to quickly generate a complete target 3D model reflecting the true shape of a target object based on a first 3D model and a second 3D model. The target 3D model has high accuracy and good rendering of the region of interest of the target object. The method proposed in this application can be applied to desktop scanners or handheld scanners, as well as to scanners such as dental scanners, facial scanners, industrial scanners, and professional scanners. It can achieve 3D reconstruction of objects or scenes such as teeth, faces, bodies, industrial products, industrial equipment, cultural relics, artworks, prostheses, medical devices, and buildings.

[0040] Referring to Figure 1, which is a flowchart illustrating a model generation method provided in some embodiments of this application, the model generation method is applied in an electronic device. The method illustrated in Figure 1 includes one or more steps, but does not constitute a limitation of this application. Furthermore, the order of the steps in the method is merely an example, and the order of the steps can be changed. Additional steps can be added or steps can be reduced without departing from the disclosure of this application. The method includes the following steps.

[0041] S101, Obtain the first and second three-dimensional models of the target object.

[0042] The first 3D model and the second 3D model are two models of the target object in different forms, or they can be two models of the target object in different time states.

[0043] In some embodiments of this application, a plaster model and an impression model of the target object are prepared, and the plaster model and the impression model of the target object are scanned using a scanner to obtain a digitized three-dimensional plaster model and a three-dimensional impression model, respectively. In this case, the first three-dimensional model is a three-dimensional plaster model, and the second three-dimensional model is a three-dimensional impression model. The target object can be a patient's oral cavity or an industrial component; this application does not limit the type of target object.

[0044] The first three-dimensional model includes three-dimensional point cloud data (or three-dimensional points) of plaster. Referring to Figure 2, a schematic diagram of the first three-dimensional model provided in some embodiments of this application is shown. Figure 2(a) shows a front view of the first three-dimensional model, and Figure 2(b) shows a back view of the first three-dimensional model. In some embodiments of this application, the second three-dimensional model includes three-dimensional points of the impression. Referring to Figure 3, a schematic diagram of the second three-dimensional model provided in some embodiments of this application is shown.

[0045] In some embodiments of this application, a scanner can simultaneously scan the actual target object to obtain a digitized actual three-dimensional model, and create an impression model of the target object. The scanner is then used to scan the impression model of the target object to obtain digitized three-dimensional impression models. In this case, the first three-dimensional model is the actual three-dimensional model, and the second three-dimensional model is the three-dimensional impression model. For example, the first three-dimensional model is a three-dimensional oral cavity model, and the second three-dimensional model is the three-dimensional impression model.

[0046] In some embodiments of this application, two digitized three-dimensional models can be obtained by scanning the target object at different times. For example, scanning the oral cavity of a patient at different orthodontic stages, or the oral cavity of a patient before and after surgery (such as the oral cavity of a patient before and after tooth extraction, the oral cavity of a patient before and after implantation, etc.), in this case, the first three-dimensional model and the second three-dimensional model can be two three-dimensional oral cavity models at different times.

[0047] S102, Determine the region of interest in the second three-dimensional model.

[0048] In some embodiments of this application, determining the region of interest in the second three-dimensional model includes: determining the marking mode selected by the user; if the marking mode is a first mode, marking the three-dimensional points in the second three-dimensional model where the user performs operations using a preset graphic; if the marking mode is a second mode, determining the range of the operation area in the second three-dimensional model where the user performs operations, and marking the three-dimensional points within the operation area; and taking the marked region formed by the marked three-dimensional points as the region of interest.

[0049] Referring to Figure 4, which is a schematic diagram of a first interface provided in some embodiments of this application, the first interface 30 includes a first control 31, a second control 32, and a second three-dimensional model 33. The first control 31 is used to set the marking mode to a first mode, and the second control 32 is used to set the marking mode to a second mode.

[0050] In some embodiments of this application, when the user sets the marking mode to the first mode via the first control 31, a preset graphic is used to mark the 3D points in the second 3D model where the user operates. The preset graphic can be spherical. For example, referring to Figure 4, if the user operates on the 3D point at position A in the second 3D model 33, the 3D point at position A can be marked based on a sphere, and the marked area formed by the marked 3D points can be used as the region of interest. It should be noted that this application does not limit the preset graphic; for example, the preset graphic can also be a circle, an ellipse, or a polygon, thus allowing the marking of 3D points in the second 3D model 33 to be based on various different preset graphics.

[0051] In some embodiments of this application, when the user sets the marking mode to the second mode via the second control 32, the operation area range in which the user performs operations in the second three-dimensional model is determined, and the three-dimensional points within the operation area range are marked. For example, if the user performs a circle operation at position A in the second three-dimensional model 33, the operation area range corresponding to the circle operation is determined to be a circular area, and the three-dimensional points within the circular area are marked, and the marked area formed by the marked three-dimensional points is taken as the region of interest.

[0052] Referring again to Figure 4, the first interface 30 also includes a third control 34. The third control 34 is used to switch the marking mode to the erasing mode. When the user switches the marking mode to the erasing mode using the third control 34, the user-marked 3D points are deleted, making it easier for the user to modify the marked 3D points.

[0053] In some other embodiments of this application, determining the region of interest in the second three-dimensional model includes: determining the region of interest in the second three-dimensional model using a preset AI model. Specifically, the region where the target component is located in the second three-dimensional model is determined using a preset AI model, and the region of the target component is taken as the region of interest. In the field of dentistry, the target component includes at least one of teeth, gums, post and core, and dental implants; in other fields, the target component includes holes, industrial equipment, buildings, etc.

[0054] For example, feature extraction and matching, or threshold segmentation, can be used to determine the data corresponding to the region of interest from the projected views of the second 3D model from various perspectives or from image frames acquired by the scanner. The recognition results are then back-projected or associated with the 3D model based on correspondences. The region of interest can be recorded in the scanning order. For instance, semantic recognition can be performed on the projected views of the second 3D model from various perspectives or from image frames acquired by the scanner. The recognition results are then back-projected or associated with the 3D model based on correspondences. In a dental scanning scenario, if the scanning order records the region of interest as tooth 1, AI recognition can be used to determine the data corresponding to tooth 1 from the projected views of the second 3D model from various perspectives or from image frames acquired by the scanner. Similarly, in an industrial scanning scenario, if the scanning order records the region of interest as industrial equipment A, AI recognition can be used to determine the data corresponding to industrial equipment A from the projected views of the second 3D model from various perspectives or from image frames acquired by the scanner. Likewise, in a large-scale building scene scanning scenario, if the scanning order records the region of interest as building 1, AI recognition can be used to determine the data corresponding to building 1 from the projected views of the second 3D model from various perspectives or from image frames acquired by the scanner.

[0055] In some other embodiments of this application, semantic recognition can also be performed on the second three-dimensional model using a preset AI model, and the region where the identified target component is located can be taken as the region of interest.

[0056] S103, Expand the region of interest and determine the area to be stitched in the second 3D model.

[0057] In some embodiments of this application, expanding the region of interest (ROI) to determine the region to be stitched in the second 3D model includes: extracting the bounding box of the ROI; expanding the bounding box outward according to a preset range to obtain an expanded bounding box; and determining the region to be stitched based on the expanded bounding box. In some embodiments of this application, the bounding box represents the region formed by the 3D points of the ROI in the second 3D model. After expanding the bounding box outward according to the preset range to obtain the expanded bounding box, the region formed by all the 3D points in the expanded bounding box is taken as the region to be stitched. It should be noted that the expanded bounding box can be a rectangular region or a circular region; this application does not limit the graphic shape of the expanded bounding box. This application can assist in stitching the first 3D model and the second 3D model by using the region to be stitched, thereby improving the success rate of stitching the first 3D model and the second 3D model.

[0058] S104: The areas to be stitched in the first 3D model and the second 3D model are stitched together to obtain stitched data, and the matching of the first 3D model and the second 3D model is determined based on the stitched data. If the first 3D model and the second 3D model are determined not to match based on the stitched data, proceed to step S105; if the first 3D model and the second 3D model are determined to match based on the stitched data, proceed to step S106.

[0059] In some embodiments of this application, the second 3D model displays a region to be stitched. Based on the region to be stitched, a comparison region in the first 3D model corresponding to the region to be stitched can be determined; a matching value between the comparison region and the region to be stitched can be calculated, where the matching value represents the degree of matching between the comparison region and the region to be stitched; if the matching value is less than a preset threshold, it is determined that the first 3D model and the second 3D model do not match; if the matching value is greater than or equal to the preset threshold, it is determined that the first 3D model and the second 3D model match.

[0060] In some embodiments of this application, the matching value may be a similarity value, and the preset threshold corresponding to the similarity value is a similarity threshold. Referring to Figure 5, a flowchart of a method for matching a first three-dimensional model and a second three-dimensional model provided in some embodiments of this application is shown, including the following steps.

[0061] S501, calculate the similarity between the comparison area and the area to be spliced ​​to obtain the similarity value.

[0062] S502, determine whether the similarity value is less than the similarity threshold. If the similarity value is less than the similarity threshold, proceed to step S503; if the similarity value is greater than or equal to the similarity threshold, proceed to step S504.

[0063] S503, It is determined that the first 3D model and the second 3D model do not match.

[0064] S504, determine the match between the first 3D model and the second 3D model.

[0065] In some embodiments of this application, one of the following algorithms can be used to calculate the similarity between the comparison region and the region to be spliced: cosine similarity algorithm, mean squared error similarity algorithm, structural similarity algorithm, and histogram algorithm. The similarity threshold can be set according to the user's needs.

[0066] For example, a cosine similarity algorithm is used to calculate the cosine of the vector angle between the comparison region and the region to be stitched together, and this cosine value is used as the similarity value to determine whether the similarity value is less than a similarity threshold (e.g., set to 0.8). If the cosine of the vector angle is less than 0.8, it is determined that the first 3D model and the second 3D model do not match; if the cosine of the vector angle is greater than or equal to 0.8, it is determined that the first 3D model and the second 3D model match.

[0067] In some embodiments of this application, the matching value may also be an overlap value, and the preset threshold corresponding to the overlap value is the overlap threshold. Referring to Figure 6, a flowchart of a method for matching a first three-dimensional model and a second three-dimensional model provided in other embodiments of this application is shown, including the following steps.

[0068] S601, calculate the overlap value between the comparison area and the area to be spliced.

[0069] S602, determine whether the overlap value is less than the overlap threshold. If the overlap value is less than the overlap threshold, proceed to step S603; if the overlap value is greater than or equal to the overlap threshold, proceed to step S604.

[0070] S603, It is determined that the first 3D model and the second 3D model do not match.

[0071] S604, determine the match between the first 3D model and the second 3D model.

[0072] In some embodiments of this application, an algorithm for calculating the percentage overlap of rectangular frames can be used to calculate the degree of overlap between the comparison area and the area to be stitched. The degree of overlap threshold can be set according to the user's needs.

[0073] For example, the intersection and union areas of the comparison region and the region to be stitched are calculated, and the Intersection over Union (IoU) value is calculated based on the intersection and union areas. It is then determined whether the IoU value is less than the overlap threshold (e.g., set to 0.7). If the IoU value is less than 0.7, the first 3D model and the second 3D model are determined to be mismatched; if the IoU value is greater than or equal to 0.8, the first 3D model and the second 3D model are determined to be matched.

[0074] In other embodiments of this application, the matching of the first 3D model and the second 3D model can also be determined manually, including: issuing a first prompt message to prompt the user to confirm whether the first 3D model and the second 3D model match; if the user's confirmation operation is received, the first 3D model and the second 3D model are determined to match; if the user's confirmation operation is not received, the first 3D model and the second 3D model are determined to not match. In embodiments of this application, the first prompt message and the control corresponding to the confirmation operation can be displayed in a pop-up window. The user inputs a confirmation operation by operating the control on the pop-up window. If the user's confirmation operation input through the pop-up window is received, the first 3D model and the second 3D model are determined to match; if the user's confirmation operation input through the pop-up window is not received, the first 3D model and the second 3D model are determined to not match.

[0075] S105, set a preset number of splicing points on the area to be spliced, and splice the first three-dimensional model with the area to be spliced ​​based on the multiple splicing points.

[0076] If the first 3D model and the second 3D model are determined to be mismatched based on the stitching data, step S105 is executed. In some embodiments of this application, a preset number of stitching points are set on the area to be stitched, and the first 3D model is stitched with the area to be stitched based on multiple stitching points, including: displaying the area to be stitched and issuing a second prompt message to allow the user to select points within the area to be stitched; if the user is detected to be selecting points within the area to be stitched, responding to the selection operation and determining the multiple points selected by the user as multiple stitching points; stitching the first 3D model with the area to be stitched based on the multiple stitching points; if the user is not detected to be selecting points within the area to be stitched, not using the multiple points selected by the user as stitching points, and issuing feedback information to allow the user to select points within the area to be stitched. In this way, this application can ensure that the user can set a preset number of stitching points on the area to be stitched and complete the stitching of the first 3D model with the area to be stitched, ensuring that there is a sufficient common area between the first 3D model and the second 3D model for stitching, thereby improving the stitching accuracy.

[0077] In some embodiments of this application, when displaying the area to be spliced, the area to be spliced ​​can be marked in a preset manner, such as by color or by a box. This makes it convenient for users, especially inexperienced doctors or technicians, to select points in the marked area to be spliced, thereby ensuring the accuracy of the selection operation.

[0078] Referring to Figure 7, which is a schematic diagram of a second interface provided in some embodiments of this application, the second interface 70 displays a first 3D model 71, a region to be stitched 72, a stitching control 73, and prompt information (not shown in the figure). The prompt information is used to remind the user to select points within the region to be stitched. If the user is detected to be selecting points within the region to be stitched 72 displayed on the second interface 70, the interface responds to the selection operation and determines multiple stitching points 74. After setting the stitching points 74, the interface receives the user's operation on the stitching control 73 and stitches the first 3D model 71 with the region to be stitched 72 based on the stitching points 74. If the user is detected not to be selecting points within the region to be stitched 72 displayed on the second interface 70, the interface sends feedback information to prompt the user to select points within the region to be stitched 72.

[0079] S106. Based on the stitching data, the first three-dimensional model and the second three-dimensional model are fused to obtain the target three-dimensional model.

[0080] If the first 3D model and the second 3D model are determined to match based on the stitching data, step S106 is executed. Referring to Figure 8, a flowchart of a method for fusing the first 3D model and the second 3D model in some embodiments of this application is shown. The method includes the following steps.

[0081] S801, Delete the portion of the area to be stitched from the data, excluding the region of interest.

[0082] In some embodiments of this application, the stitching data includes 3D points of a first 3D model and 3D points of a second 3D model. The portion of the region to be stitched, excluding the region of interest, can be deleted from all 3D points in the stitching data.

[0083] It should be noted that the deletion operation of the part other than the region of interest in the area to be stitched from the stitching data in the embodiments of this application has multiple implementation schemes, including but not limited to: one implementation scheme is to separate the part other than the region of interest in the area to be stitched from the stitching data, and only retain the stitched data after separating the part other than the region of interest in the area to be stitched in the display interface. In this implementation scheme, the deletion operation is a selective retention of the display result, rather than completely deleting the part other than the region of interest in the area to be stitched, that is, only deleting the part other than the region of interest in the area to be stitched from the display interface; another implementation scheme is to directly and completely delete the part other than the region of interest in the area to be stitched from the source data; yet another implementation scheme is to separate the part other than the region of interest in the area to be stitched from the stitching data and increase the transparency of the separated part other than the region of interest in the area to be stitched, such as semi-transparency display, to achieve the effect of highlighting the remaining stitched data on the display interface; yet another implementation scheme is to delete the part other than the region of interest in the area to be stitched from the intermediate product of the reconstruction model or the reconstructed model, so that the display interface does not display the part other than the region of interest in the area to be stitched.

[0084] S802, perform an overlap removal operation on the spliced ​​data to obtain the initial fusion model.

[0085] In some embodiments of this application, the process of performing a de-overlapping operation on the stitched data to obtain an initial fusion model includes: assigning correlation values ​​to the three-dimensional points in the stitched data, wherein the correlation value represents the correlation between the three-dimensional points and the initial fusion model; performing a de-overlapping operation on the three-dimensional points in the stitched data based on the correlation values ​​of the three-dimensional points to obtain the initial fusion model; wherein the correlation value of the three-dimensional points in the region of interest in the second three-dimensional model is the highest, and the correlation value of the three-dimensional points in the first three-dimensional model is greater than the correlation value of the three-dimensional points in the second three-dimensional model.

[0086] In some embodiments of this application, the correlation value can be represented as a priority, where a higher priority results in a larger correlation value, and a lower priority results in a smaller correlation value. In other embodiments of this application, the correlation value can be represented as a weight value, where a larger weight value results in a higher correlation value, and a smaller weight value results in a smaller correlation value. This application effectively removes overlapping 3D points by performing a de-overlapping operation on the 3D points in the stitched data based on the correlation value of the 3D points, and can retain the 3D points of the target components (such as pile cores) in the initial fusion model.

[0087] S803 performs data cleanup on the initial fused model to obtain the target 3D model.

[0088] In some embodiments of this application, the de-noising data operation on the initial fusion model includes: deleting target data blocks in the initial fusion model, wherein the number of 3D points in the target data blocks is less than a preset number. After removing the target data blocks from the initial fusion model, a target 3D model is obtained (refer to Figure 9). Deleting target data blocks with fewer than a preset number of 3D points in this application can effectively remove useless noisy data from the target 3D model.

[0089] This application uses the region of interest (ROI) as a reference. When the ROI has the highest priority, it can effectively remove miscellaneous data from the first 3D model of a single jaw, ensuring a better final merged model. Furthermore, this application eliminates the need for manual editing and deletion of miscellaneous data from the scanned first and second 3D models of a single jaw. It only requires marking the ROI to be retained (such as "stub" data within the ROI) before deletion, thus avoiding accidental data deletion and reducing the possibility of operational errors, especially from inexperienced doctors or technicians who might accidentally delete data, resulting in an incomplete target 3D model.

[0090] In some embodiments of this application, before step S104, the model generation method further includes: flipping the second three-dimensional model along a preset direction so that the area to be spliced ​​in the second three-dimensional model is flipped, or only the area to be spliced ​​is flipped along the preset direction, wherein the preset direction is the normal of the area to be spliced. Wherein, if the first three-dimensional model is a three-dimensional plaster model and the second three-dimensional model is a three-dimensional impression model, or the first three-dimensional model is a three-dimensional oral cavity model and the second three-dimensional model is a three-dimensional impression model. The second three-dimensional model and the first three-dimensional model are mutually concave-convex. Since the second three-dimensional model and the first three-dimensional model are mutually concave-convex, flipping the second three-dimensional model so that the area to be spliced ​​in the second three-dimensional model is normally flipped makes the normal of the first three-dimensional model the same as the normal of the modified second three-dimensional model, thereby facilitating the splicing of the first three-dimensional model with the area to be spliced.

[0091] This application expands the region of interest (ROI) of the second 3D model to determine the region to be stitched within the second 3D model. It then stitches the first 3D model with the region to be stitched in the second 3D model to obtain stitched data. When the two models are determined to match based on the stitched data, they are fused to obtain a complete target 3D model. This allows for the rapid generation of a complete target 3D model that accurately reflects the true shape of the target object, based on both the first and second 3D models. This target 3D model exhibits high accuracy and effectively presents the ROI, avoiding the problems of existing target 3D models failing to accurately represent the ROI or unclear scanning of the first 3D model, which hinders the clear scanning of the complete target 3D model. Furthermore, by expanding the ROI to obtain the region to be stitched and stitching the first and second 3D models based on this region, this application ensures sufficient common area between the two models for stitching, thereby improving stitching accuracy. This application also removes the portion of the region to be stitched from the stitching data between the first 3D model and the second 3D model, excluding the region of interest, and performs a de-overlapping point operation on the stitching data to obtain an initial fused model. This can effectively reduce the overlapping data of the initial fused model of the two models. Thus, there is no need to edit or delete the overlapping data in the first 3D model and the second 3D model, which simplifies the generation process of the target 3D model, improves the design efficiency of the target 3D model, and reduces the error rate of the user's deletion operation.

[0092] Referring to Figure 10, this is a structural diagram of a model generation apparatus provided in some embodiments of this application. The model generation apparatus can implement the details of the model generation method in the above embodiments and achieve the same effect. The model generation apparatus 90 includes the following modules.

[0093] Module 901 acquires the first and second 3D models of the target object.

[0094] The first determining module 902 is used to determine the region of interest in the second three-dimensional model.

[0095] The second determining module 903 is used to expand the region of interest and determine the region to be stitched in the second three-dimensional model.

[0096] The first stitching module 904 stitches the areas to be stitched in the first 3D model and the second 3D model to obtain stitching data, and determines whether the first 3D model and the second 3D model match based on the stitching data.

[0097] The second splicing module 905 is used to set a preset number of splicing points on the area to be spliced ​​when the first three-dimensional model and the second three-dimensional model do not match, and to splice the first three-dimensional model and the area to be spliced ​​based on the multiple splicing points.

[0098] The fusion module 906, when determining the match between the first 3D model and the second 3D model, fuses the first 3D model and the second 3D model according to the stitching data to obtain the target 3D model.

[0099] This application expands the region of interest (ROI) of the second 3D model to determine the region to be stitched within the second 3D model. It then stitches the first 3D model with the region to be stitched in the second 3D model to obtain stitched data. When the two models are determined to match based on the stitched data, they are fused to obtain a complete target 3D model. This allows for the rapid generation of a complete target 3D model that accurately reflects the true shape of the target object, based on both the first and second 3D models. This target 3D model exhibits high accuracy and effectively presents the ROI, avoiding the problems of existing target 3D models failing to accurately represent the ROI or unclear scanning of the first 3D model, which hinders the clear scanning of the complete target 3D model. Furthermore, this application expands the ROI to obtain the region to be stitched and stitches the first and second 3D models based on this region, ensuring sufficient common area between the two models for stitching, thereby improving stitching accuracy.

[0100] It should be noted that the model generation device provided in the above embodiments is only an example of the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0101] The functional units and modules in the above embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of the embodiments of this application.

[0102] This application also provides an electronic device. Referring to FIG11, which is a schematic diagram of an electronic device provided in some embodiments of this application, the electronic device 10 includes: a memory 11, a processor 12, and a computer program 13 stored in the memory 11 and executable on the processor 12. When the processor 12 executes the computer program, it implements the steps in the various method embodiments described above.

[0103] Electronic device 10 can be a general-purpose electronic device or a dedicated electronic device. In specific embodiments, electronic device 10 can be a desktop computer, portable computer, network server, handheld computer, mobile phone, tablet computer, wireless terminal device, communication device, or embedded device. The embodiments of this application do not limit the type of electronic device 10. Those skilled in the art will understand that FIG10 is merely an example of electronic device 10 and does not constitute a limitation on electronic device 10. It may include more or fewer components than shown, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0104] Processor 12 can be a Central Processing Unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0105] In some embodiments, memory 11 may be an internal storage unit of electronic device 10, such as a hard disk or RAM of electronic device 10. In other embodiments, memory 11 may also be an external storage device of electronic device 10, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on electronic device 10. Furthermore, memory 11 may include both internal and external storage units of electronic device 10. Memory 11 is used to store operating system, applications, boot loader, data, and other programs. Memory 11 may also be used to temporarily store data that has been output or will be output.

[0106] Based on the same concept, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0107] The aforementioned computer-readable storage medium can be any medium capable of storing program code, such as Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), and Electrically Erasable Programmable Read-Only Memory (EEPROM). The storage medium stores the program, and after receiving execution instructions, the processor executes the program. The method executed by the electronic device defined by the process disclosed in any embodiment of this invention can be applied to the processor or implemented by the processor.

[0108] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0109] It should be understood that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. Their specific functions and technical effects can be found in the method embodiments section, and will not be repeated here. For ease of description and brevity, only the division of the above-mentioned functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0110] It can be replaced and can be implemented, wholly or partially, through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented, wholly or partially, in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated.

[0111] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application. Industrial applicability

[0112] The model generation scheme provided in this application, when determining that two models match based on the stitching data of the first and second 3D models, fuses the first and second 3D models to obtain a complete target 3D model. This allows for the rapid generation of a complete target 3D model reflecting the true shape of the target object based on the first and second 3D models. Furthermore, this target 3D model has high accuracy and good rendering of the region of interest (ROI) of the target object. This avoids the problem of existing target 3D models failing to accurately reflect the ROI or having unclear scans of the first 3D model, resulting in the inability to generate a complete 3D model reflecting the true shape of the object. In addition, this application expands the ROI to obtain the stitching region and stitches the first and second 3D models based on this region, ensuring sufficient common area between the first and second 3D models for stitching, thereby improving stitching accuracy. This application also removes the portion of the region to be stitched from the stitching data between the first 3D model and the second 3D model, excluding the region of interest, and performs a de-overlapping point operation on the stitching data to obtain an initial fused model. This effectively reduces the overlapping data in the initial fused model of the two models. Thus, there is no need to edit or delete the overlapping data in the first 3D model and the second 3D model, which simplifies the generation process of the target 3D model, improves the design efficiency of the target 3D model, and reduces the error rate of user deletion operations, making it highly practical for industrial applications.

Claims

1. A model generation method, wherein, The method includes: Obtain the first and second 3D models of the target object; Determine the region of interest in the second 3D model; Expand the region of interest to determine the region to be stitched in the second 3D model; The first three-dimensional model and the area to be stitched in the second three-dimensional model are stitched together to obtain stitched data, and the first three-dimensional model and the second three-dimensional model are matched based on the stitched data. When determining that the first 3D model matches the second 3D model, the first 3D model and the second 3D model are fused together according to the stitching data to obtain the target 3D model.

2. The model generation method as described in claim 1, wherein, The step of fusing the first 3D model and the second 3D model according to the stitched data to obtain the target 3D model includes: Delete the portion of the region to be spliced ​​from the spliced ​​data, excluding the region of interest; The spliced ​​data is subjected to a de-overlapping operation to obtain an initial fusion model; The initial fusion model is subjected to a data cleanup operation to obtain the target 3D model.

3. The model generation method as described in claim 2, wherein, The process of removing overlapping points from the spliced ​​data to obtain the initial fusion model includes: Assign a correlation value to the three-dimensional points in the stitched data, wherein the correlation value represents the correlation between the three-dimensional points and the initial fusion model; The initial fusion model is obtained by performing a de-overlapping operation on the three-dimensional points in the stitched data based on the correlation degree value of the three-dimensional points. Among them, the correlation value of the three-dimensional points in the region of interest in the second three-dimensional model is the highest, while the correlation value of the three-dimensional points in the first three-dimensional model is greater than that of the three-dimensional points in the second three-dimensional model.

4. The model generation method as described in claim 2, wherein, The data cleanup operation on the initial fusion model includes: Delete the target data block in the initial fusion model, wherein the number of three-dimensional points included in the target data block is less than a preset number.

5. The model generation method as described in claim 1, wherein, Determining the region of interest in the second three-dimensional model includes: Determine the tagging pattern selected by the user; If the marking mode is the first mode, the three-dimensional points in the second three-dimensional model where the user operates are marked using a preset graphic; If the marking mode is the second mode, determine the range of the operation area where the user performs operations in the second three-dimensional model, and mark the three-dimensional points within the operation area; The region of interest is defined as the area formed by the marked three-dimensional points.

6. The model generation method as described in claim 1, wherein, Determining the region of interest in the second three-dimensional model includes: The region of interest in the second 3D model is determined by a preset AI model.

7. The model generation method as described in claim 1, wherein, Expanding the region of interest and determining the region to be stitched in the second 3D model includes: Extract the bounding box of the region of interest, and expand the bounding box outward according to a preset range to obtain an expanded bounding box; Based on the outer bounding box, the area to be spliced ​​is determined.

8. The model generation method as described in claim 1, wherein, The first three-dimensional model is a three-dimensional plaster model, and the second three-dimensional model is a three-dimensional impression model, or the first three-dimensional model is a three-dimensional oral cavity model, and the second three-dimensional model is a three-dimensional impression model.

9. The model generation method as described in claim 1, wherein, The second 3D model and the first 3D model are mutually concave-convex matched. Before stitching the areas to be stitched in the first 3D model and the second 3D model, the method further includes: The second three-dimensional model is flipped along a preset direction so that the area to be stitched in the second three-dimensional model is flipped, wherein the preset direction is the normal of the area to be stitched.

10. The model generation method as described in claim 1, wherein, When it is determined that the first 3D model and the second 3D model do not match, the method further includes: Display the area to be stitched and issue a first prompt message so that the user can select points within the area to be stitched; If a user is detected to be selecting points within the area to be stitched, respond to the selection operation and determine multiple stitching points; The first 3D model is stitched together with the area to be stitched based on the multiple stitching points; If it is detected that the user is not performing a point selection operation within the area to be stitched, a feedback message is sent so that the user can perform a point selection operation within the area to be stitched.

11. The model generation method as described in claim 1, wherein, The step of determining whether the first 3D model and the second 3D model match based on the stitched data includes: Determine the comparison region in the first 3D model that corresponds to the region to be stitched together; Calculate the matching value between the comparison region and the region to be spliced, where the matching value represents the degree of matching between the comparison region and the region to be spliced; If the matching value is less than a preset threshold, it is determined that the first 3D model and the second 3D model do not match. If the matching value is greater than or equal to the preset threshold, it is determined that the first three-dimensional model and the second three-dimensional model are matched.

12. The model generation method as described in claim 1, wherein, The step of determining whether the first 3D model and the second 3D model match based on the stitched data includes: A second prompt message is issued to ask the user to confirm whether the first 3D model and the second 3D model match; If a user's confirmation is received, it is determined that the first 3D model matches the second 3D model; If no confirmation is received from the user, it is determined that the first 3D model and the second 3D model do not match.

13. An electronic device, wherein, include: A processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the electronic device performs the model generation method according to any one of claims 1 to 12.

14. A computer-readable storage medium having a computer program stored thereon, wherein, When the computer program is executed by a processor, it implements the model generation method as described in any one of claims 1 to 12.

15. A model generation apparatus, wherein, The model generation device includes: The acquisition module is used to acquire the first and second 3D models of the target object. The first determining module is used to determine the region of interest in the second three-dimensional model; The second determining module is used to expand the region of interest and determine the region to be stitched in the second three-dimensional model. The first stitching module is used to stitch together the areas to be stitched in the first three-dimensional model and the second three-dimensional model to obtain stitching data, and to determine whether the first three-dimensional model and the second three-dimensional model match based on the stitching data; The fusion module is used to fuse the first three-dimensional model and the second three-dimensional model according to the stitching data when it is determined that the first three-dimensional model and the second three-dimensional model match, so as to obtain the target three-dimensional model.

Images