Defect repairing method for three-dimensional reconstruction model of porous structure

Abstract

The invention relates to the technical field of generative image restoration, in particular to a defect restoration method for a three-dimensional reconstruction model of a porous structure, which comprises the following steps: generating a basic three-dimensional model of a target object according to a target resolution; scanning a target object and determining a corresponding to-be-repaired area; according to the target depth of the to-be-repaired area, determining a corresponding repair focus and a repair resolution; generating a repair container, and placing the geometric center of the repair container on the repair focus; filling the repair container with the repair resolution according to a preset sequence, and generating a plurality of repair models; identifying feature points of each repair model and the basic three-dimensional model, and merging the repair models into the basic three-dimensional model according to the similarity of the feature points; according to the method, the corresponding repairing features are selected, the model for repairing is merged into the target model according to the features, and edge smoothing processing is performed, so that the integrity of the model with the holes is effectively improved while the quality of the model is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of generative image restoration, and in particular to a method for repairing defects in a three-dimensional reconstruction model of a porous structure. Background Art

[0002] The scanning beams used in existing 3D scanning technologies are mainly laser and structured light. 3D scanning technology determines the spatial position information of the object surface by emitting a scanning beam and receiving reflected light, and then constructs a 3D model. However, when processing holes inside an object, after the scanning beam enters the hole, due to multiple reflections and refractions on the wall inside the hole and the attenuation of the scanning beam energy, the reflected light signal becomes extremely complex and weak, making it difficult for the scanner to accurately capture and analyze these signals, and thus unable to fully obtain the structural information inside the hole.

[0003] Especially for some deep holes, micro-holes or holes with complex and irregular internal structures, the limitations of laser scanning technology are more obvious. For example, when scanning some objects with fine internal micro-hole structures, laser scanning can often only obtain partial information about the entrance of the micro-hole, while it is difficult to detect information such as the depth, curvature and branch structure inside the micro-hole. As a result, the 3D model scanned by the existing 3D scanner cannot obtain its surface model information when encountering a deep or small hole in the scanned object, and the scanned 3D model is incomplete.

[0004] Chinese Patent Authorization Announcement No.: CN118823252B discloses a method, device, equipment and storage medium for reconstructing a three-dimensional model, the method comprising: performing multi-directional image acquisition on an object to be reconstructed to obtain multiple images of the object to be reconstructed; inputting each of the images of the object to be reconstructed into a trained de-reflection model for de-reflection processing to obtain multiple de-reflection reconstructed object images; wherein the de-reflection model sequentially performs multiple down-sampling and multiple up-sampling on the input image, and fuses the result of each up-sampling and the input image through channel splicing and transposed convolution to obtain a de-reflection image; using a pre-trained de-occlusion model, based on the occlusion mask map of the occlusion area in each of the de-reflection reconstructed object images, de-occlusion processing is performed on the occlusion area of ​​each of the de-reflection reconstructed object images to obtain multiple processed reconstructed object images; using each of the processed reconstructed object images to perform three-dimensional reconstruction to obtain a three-dimensional model of the object to be reconstructed.

[0005] It is not difficult to see that the above solution has the problem that it is difficult to effectively generate and accurately repair the model when the model being processed has many concave surfaces. Summary of the invention

[0006] To this end, the present invention provides a method for repairing defects in a three-dimensional reconstruction model of a porous structure, so as to overcome the problem in the prior art that when scanning some objects with fine internal micropore structures, laser scanning can often only obtain partial information about the entrance of the micropores, while information such as the depth, curvature and branch structure inside the micropores is difficult to detect, thereby resulting in an incomplete scanned three-dimensional model.

[0007] To achieve the above objectives, on the one hand, the present invention provides a method for repairing defects in a three-dimensional reconstruction model of a porous structure, comprising: Generate a basic 3D model of the target object at a target resolution; Scan the target object and determine the corresponding area to be repaired; Determine the corresponding repair focus and repair resolution according to the target depth of the area to be repaired; Generate a repair container, and place the geometric center of the repair container on the repair focus; Filling the repair container with the repair resolution in a preset order and generating a plurality of repair models; Identify feature points of each repair model and the basic three-dimensional model, and merge the repair model into the basic three-dimensional model according to the similarity of the feature points; Wherein, the repair container wraps at least one characteristic point.

[0008] Further, the step of determining the area to be repaired includes: Pre-segmenting the basic three-dimensional model based on the size of the repair container to form a plurality of segmentation lines; By comparing the basic three-dimensional model, determining the position of each hole in the three-dimensional model; Adjusting the segmentation line to adjust the position of each hole to the complete area of ​​the basic three-dimensional model; The basic three-dimensional model is segmented according to the segmentation line to form a corresponding area to be repaired.

[0009] Furthermore, the step of generating the repair focus includes: determining a maximum hole depth in the repair container; Determining the visible hole depth according to the maximum hole depth; Based on the visible hole depth, the geometric center of the visible hole depth is selected as the repair focus.

[0010] Furthermore, the step of determining the depth of the visible holes comprises: determining a container outer surface of the repaired container and a model outer surface of the basic three-dimensional model contained in the repaired container; Selecting holes on the outer surface of the model, and taking the geometric center of the repaired container as a reference, projecting the holes on the outer surface of the container to form a shooting reference plane; The hole corresponding to the shooting reference plane is photographed at the target resolution, and the depth that can be resolved at the target resolution is recorded as the visible hole depth.

[0011] Furthermore, the step of selecting the geometric center of the visible hole depth as the repair focus includes: Generate a corresponding inner surface based on the visible hole depth; The geometric center of the hole projected on the inner surface is used as the repair focus.

[0012] Furthermore, the process of filling the repair container according to the repair resolution includes: According to the restoration resolution, nesting the restoration container on the basic three-dimensional model; Adjusting the angle of the repair container so that any one of its surfaces is parallel to the average plane of the area to be repaired; Adjusting the lens position corresponding to the restoration resolution to adjust the size of the restoration container; The repair container is adjusted so that the projection of any surface of the repair container on the average plane can wrap the projection of the area to be repaired on the average plane.

[0013] Furthermore, for a single repair model, the process of selecting its feature points includes: Identify and compare several geometric elements in the repair model and the base model; Superimpose the same geometric elements, and superimpose the remaining positions of the repaired model with the base model; The geometric elements whose overlapping degree is not less than a preset overlapping degree are recorded as feature points of the repair model.

[0014] Furthermore, the pixels corresponding to the repair resolution are not less than 2 times of the target resolution.

[0015] On the other hand, the present invention also provides a method for repairing defects in a three-dimensional reconstruction model of a porous structure, comprising: Generate a basic 3D model of the target object at a target resolution; Scan the target object and determine the corresponding area to be repaired; Determine the corresponding repair focus and repair resolution according to the target depth of the area to be repaired; Generate a plurality of repair containers, and place the geometric centers of the repair containers on the repair focus; Filling the repair container with the repair resolution in a preset order and generating a plurality of repair models; Identify feature points of each repair model and the basic three-dimensional model, and merge each repair model into the basic three-dimensional model; Wherein, the repair containers correspond to single feature points respectively.

[0016] Furthermore, for a single feature point, a model with a smooth interface transition is selected as the repair model of the feature point; When the repaired container corresponding to each feature point is merged into the basic three-dimensional model, the interface between the corresponding repaired model and the basic three-dimensional model is also smoothed.

[0017] Compared with the prior art, the beneficial effect of the present invention lies in that the target model is scanned with different resolutions, and the more refined modeling is used as the repair model, the corresponding repair features are selected, the repair model is merged into the target model according to the features, and the edge smoothing is performed, which improves the quality of the model while effectively improving the integrity of the model with holes.

[0018] Furthermore, by setting up a repair container, the area to be repaired can be isolated, thereby reducing the waste of computing power caused by loading the complete model, and more computing power resources can be invested in generating the repair model. While effectively improving the efficiency of the repair model, it can also improve the integrity of the model with holes.

[0019] Furthermore, by selecting the repair focus, the position of the repair container is fixed, thereby effectively avoiding the problem of displacement of the repair model caused by errors in the direction and / or position of the generated repair model, effectively reducing the waste of computing power in model repair and improving the accuracy of repairing models with holes.

[0020] Furthermore, by setting the geometric elements in the model and superimposing the geometric elements, the repair model is superimposed on the basic model, and each model is adjusted according to the display. This effectively improves the precision of the repair of the model with holes and further improves the integrity of the model with holes. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A flow chart of a defect repair method for a three-dimensional reconstruction model of a porous structure according to the present invention; Figure 2 A flowchart of a defect repair method for a three-dimensional reconstruction model of a porous structure according to an embodiment of the present invention; Figure 3 A schematic diagram of aligning a three-dimensional model according to an embodiment of the present invention; Figure 4 A schematic diagram of a model with holes that has been repaired according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0023] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0024] It should be noted that, in the description of the present invention, terms such as "up", "down", "left", "right", "inside" and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the drawings. This is merely for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present invention.

[0025] In addition, it should be noted that in the description of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0026] The present invention aims to use a method to repair a defective 3D model when a 3D scanner is used to perform 3D scanning on an object with holes, and the deeper holes or holes on the object cannot be completely scanned due to the accuracy defects of the 3D scanner itself. This method uses an image algorithm to restore the missing part of the 3D model when the 3D scanner scans the object, thereby achieving the purpose of repairing the missing part to form a complete 3D model.

[0027] In order to obtain the information of holes inside an object without destroying the object and without other means, taking pictures from multiple angles can obtain the most effective information inside the hole.

[0028] 1. Current status and limitations of scanning modeling technology In the current gaming, industrial manufacturing, digital protection of cultural relics, architectural planning and other industries, the technology of modeling through 3D scanning has been widely used to obtain 3D information of objects. Among them, laser scanning technology is a more common one. It calculates the distance information of each point on the surface of the object by emitting a high-energy laser beam to the surface of the object, and then receives the time difference and intensity change of the reflected light to build a 3D model. For example, in industrial manufacturing, the design and inspection of automotive parts often use laser scanning technology to quickly obtain the shape contour data of the parts to ensure production accuracy. In the field of architecture, this technology is also used for the digital preservation of historical buildings, which can record the appearance of the building in detail. However, when facing the 3D modeling of holes inside objects, laser scanning technology faces many difficulties. Due to the characteristics of the laser beam, after entering the hole, it is easy to be reflected and scattered by the internal wall of the hole and the rapid attenuation of energy, which makes it difficult to accurately capture and analyze the reflected signal. Especially for some deep holes, microholes or holes with complex and irregular internal structures, such as the complex cooling channels inside aircraft engine blades, laser scanning can often only obtain partial information about the entrance of the hole, and cannot deeply detect its detailed internal structure, making it difficult to build a complete and accurate three-dimensional model of the internal holes.

[0029] Structured light scanning technology projects a specific light pattern (such as stripe light, grid light, etc.) onto the surface of an object, uses a camera to record the deformation of the light pattern on the surface of the object, and calculates the three-dimensional coordinates of each point on the surface of the object based on the principle of triangulation. In the field of cultural heritage protection, for the digital modeling of some large sculptures, structured light scanning technology can obtain a high-precision surface model in a relatively short time. However, when dealing with holes inside an object, after the light is projected into the hole, it will be affected by the combined effects of occlusion, reflection and refraction inside the hole, causing the projected light pattern to be severely deformed and missing inside the hole, resulting in the inability to obtain the model surface information of the object.

[0030] At present, when reverse modeling is required, 3D scanning technology is usually used. In this process, various complex object structures are often encountered, especially when the object has special forms such as holes and holes. Due to the limited accuracy of the current 3D scanner technology, the scanning beam basically uses laser or structured light, and the scanning range and angle are greatly limited by the physical principles, and it is often difficult to scan some specific angle positions. For example, when scanning mechanical parts with complex internal cavities and fine hole structures, the scanning light of the scanner cannot penetrate into some narrow corners inside the hole, or in some parts with special curvature and occlusion relationships, it is impossible to obtain complete structural information, resulting in partial missing of the generated 3D model. This missing will not only affect the integrity and accuracy of the model, but also have a certain impact on a series of subsequent work such as product design optimization, quality inspection, virtual assembly, etc. based on the model.

[0031] The emergence of this method provides an innovative way to solve this thorny problem. It mainly uses advanced image algorithms to restore the parts that the 3D scanner cannot scan.

[0032] First, by photographing the object with holes from multiple different angles, a specified number of image data are obtained. These images cover different perspectives of the holes and information about the surrounding structures. Then, the images are deeply processed using photo modeling software to generate a three-dimensional model of the hole area of ​​the object. Finally, the generated three-dimensional model data is aligned with the original scanned model and seamlessly integrated, so that the defective three-dimensional model caused by scanning can be repaired and the final complete three-dimensional model can be generated. This method can not only effectively improve the integrity and accuracy of three-dimensional scanning modeling, but also to a certain extent break through the limitations of three-dimensional scanners in angle and structure detection, and provide a more reliable and efficient solution for the three-dimensional modeling of complex objects with holes involved in many fields such as production and manufacturing, industrial design, and cultural relics restoration, and promote the development and progress of technology in related fields.

[0033] 2. The rise and challenges of image modeling technology With the development of digital image processing technology and computer vision technology, image modeling technology has gradually emerged. Image modeling technology mainly uses a sequence of images of an object taken from different angles to reconstruct a three-dimensional model of an object. In the field of film and television production, image modeling technology is used to build some virtual scenes. By shooting multi-angle images of real scenes, and then using algorithms to generate three-dimensional models, the realism of the scene is increased. In the field of e-commerce, for the display of some products, image modeling can also be performed after shooting the products from multiple angles, so that consumers can view the products in all directions online. However, in terms of three-dimensional modeling of holes inside objects, image modeling technology also faces its own challenges. On the other hand, how to accurately determine the positional relationship of each point inside the hole in three-dimensional space from multi-angle images requires complex algorithms and a large amount of computing resources. For example, when image modeling the internal pore structure of biological bones, due to the complex internal structure of bones and the different absorption and scattering characteristics of different tissues to light, it is difficult to accurately identify and locate the hole information in the captured image. It is necessary to develop a special image processing algorithm to enhance the hole features and accurately calculate its three-dimensional coordinates.

[0034] To sum up, in terms of obtaining information about holes inside objects, pictures are slightly better than scanned models. When the physical object cannot be destroyed and there are no other means, we can only use multi-angle shooting to obtain the most effective information inside the hole, maximize the restoration of the inside of the hole, and can be used to repair the missing parts of the 3D scanned model.

[0035] See also Figure 1As shown, it is a flow chart of the defect repair method of the three-dimensional reconstruction model of a porous structure of the present invention, comprising: Step S1, generating a basic three-dimensional model of a target object according to a target resolution; Step S2, scanning the target object and determining the corresponding area to be repaired; Step S3, determining the corresponding repair focus and repair resolution according to the target depth of the area to be repaired; Step S4, generating a repair container, and placing the geometric center of the repair container on the repair focus; Step S5, filling the repair container with the repair resolution in a preset order and generating a plurality of repair models; Step S6, identifying feature points of each repaired model and the basic three-dimensional model, and merging the repaired model into the basic three-dimensional model according to the similarity of the feature points; The repair container wraps at least one feature point.

[0036] The target model is scanned at different resolutions, and the more detailed modeling is used as the repair model. The corresponding repair features are selected, and the repair model is merged into the target model according to the features, and the edges are smoothed. This improves the quality of the model while effectively improving the integrity of the model with holes.

[0037] Specifically, the steps to identify areas to be repaired include: Pre-segment the basic three-dimensional model based on the size of the repaired container to form a number of segmentation lines; By comparing with the basic three-dimensional model, determine the position of each hole in the three-dimensional model; Adjust the splitting lines to adjust the positions of the holes to the complete area of ​​the basic 3D model; The basic three-dimensional model is segmented according to the segmentation line to form a corresponding area to be repaired.

[0038] By setting up a repair container, the area to be repaired can be isolated, thereby reducing the computing power waste caused by loading the complete model and investing more computing power resources in generating the repair model. This effectively improves the efficiency of repairing the model while improving the integrity of the model with holes.

[0039] Specifically, the steps of generating the repair focus include: Determine the maximum hole depth in the repair container; Determine the visible hole depth based on the maximum hole depth; Based on the visible hole depth, the geometric center of the visible hole depth is selected as the repair focus.

[0040] Specifically, the steps to determine the visible hole depth include: determining a container outer surface of the repaired container and a model outer surface of the basic three-dimensional model contained in the repaired container; Select the holes on the outer surface of the model, and use the geometric center of the repaired container as a reference to project the holes onto the outer surface of the container to form a shooting reference plane; The hole corresponding to the reference surface is photographed at the target resolution, and the depth that can be resolved at the target resolution is recorded as the visible hole depth.

[0041] Embodiment 1: Step 1: Generate a basic 3D model of the target object at the target resolution Operation: Use a high-precision 3D scanner to scan the complete antique vase and generate a basic 3D model. The target resolution is set to 0.1 mm to ensure that the details of the model are fine enough.

[0042] Result: A complete, high-precision 3D model of the vase is obtained.

[0043] Step 2: Scan the target object and determine the corresponding area to be repaired Action: Scan the broken vase and identify the missing parts (e.g. the neck of the vase) by comparing the intact model with the broken model.

[0044] Result: The bottleneck was identified as the area to be repaired.

[0045] Step 3: Determine the corresponding repair focus and repair resolution based on the target depth of the area to be repaired Operation: Measure the depth of the bottleneck (assuming it is 2 cm) and determine the center of the bottleneck as the repair focus. The repair resolution is set to 0.1 mm to match the accuracy of the base model.

[0046] Results: The repair focus and repair resolution were determined.

[0047] Step 4: Generate a repair container and place the geometric center of the repair container on the repair focus Action: Create a virtual repair container (e.g. a cylindrical region) whose geometric center is located at the center point of the bottleneck.

[0048] Result: A repair container was generated that wrapped the bottleneck area.

[0049] Step 5: Fill the repair container with the repair resolution in the preset order and generate several repair models Operation: Fill the repair container layer by layer in order from bottom to top to generate multiple repair model fragments (for example, each layer is 0.1 mm thick).

[0050] Result: Multiple repair model fragments are generated, each corresponding to a part of the bottleneck.

[0051] Step 6: Identify the feature points of each repaired model and the basic 3D model, and merge the repaired model into the basic 3D model according to the similarity of the feature points Operation: Use feature point recognition algorithms (such as SIFT or SURF) to find similar feature points between the repaired model and the base model (such as the texture and edge of the vase), and then merge the repaired model fragments into the base model one by one.

[0052] Result: The restored vase model is highly consistent with the original model in the bottleneck part and the details are complete.

[0053] Example 2: Industrial Parts Repair Suppose a broken mechanical gear needs to be repaired.

[0054] Step 1: Generate a basic 3D model of the target object at the target resolution Operation: Based on the design drawings of the gear, use CAD software to generate a high-precision basic 3D model with a target resolution of 0.05 mm.

[0055] Result: A complete gear model is obtained.

[0056] Step 2: Scan the target object and determine the corresponding area to be repaired Action: A 3D scan of the damaged gear revealed that one of the gear tooth tips was partially missing.

[0057] Result: The tooth tip was determined to be the area to be repaired.

[0058] Step 3: Determine the corresponding repair focus and repair resolution based on the target depth of the area to be repaired Operation: Measure the depth of the tooth tip (assuming it is 5 mm), determine the center point of the tooth tip as the repair focus, and set the repair resolution to 0.05 mm.

[0059] Results: The repair focus and repair resolution were determined.

[0060] Step 4: Generate a repair container and place the geometric center of the repair container on the repair focus Action: Create a virtual repair container (e.g. a cubic region) whose geometric center is located at the center point of the tooth tip.

[0061] Result: A restorative container was created that enclosed the tooth tip area.

[0062] Step 5: Fill the repair container with the repair resolution in the preset order and generate several repair models Operation: Fill the repair container layer by layer in the order from tooth root to tooth tip to generate multiple repair model fragments (for example, each layer is 0.05 mm thick).

[0063] Result: Several restoration model fragments were generated, each corresponding to a part of the tooth tip.

[0064] Step 6: Identify the feature points of each repaired model and the basic 3D model, and merge the repaired model into the basic 3D model according to the similarity of the feature points Operation: Use feature point recognition algorithm to find similar feature points between the repair model and the base model (such as gear tooth shape, angle, etc.), and then merge the repair model fragments into the base model one by one.

[0065] Results: The repaired gear model is highly consistent with the original model in the tooth tip part, and the mechanical properties meet the requirements.

[0066] Example 3: Medical model repair Suppose you need to repair a bone model (such as a skull) that is missing parts due to an incomplete scan.

[0067] Step 1: Generate a basic 3D model of the target object at the target resolution Operation: Generate a high-precision 3D model of the skull base based on medical imaging (such as CT scan) data, with a target resolution of 0.5 mm.

[0068] Result: A complete skull model was obtained.

[0069] Step 2: Scan the target object and determine the corresponding area to be repaired Action: Analysis of the actual scanned skull revealed that the right temporal bone portion of the skull was missing.

[0070] Results: The temporal bone area was identified as the area to be repaired.

[0071] Step 3: Determine the corresponding repair focus and repair resolution based on the target depth of the area to be repaired Operation: Measure the depth of the temporal bone (assumed to be 1 cm), determine the center point of the temporal bone as the repair focus, and set the repair resolution to 0.5 mm.

[0072] Results: The repair focus and repair resolution were determined.

[0073] Step 4: Generate a repair container and place the geometric center of the repair container on the repair focus Procedure: Create a virtual restoration container (e.g., an irregularly shaped area) with its geometric center at the center point of the temporal bone.

[0074] Results: A repair vessel was generated that enclosed the temporal bone region.

[0075] Step 5: Fill the repair container with the repair resolution in the preset order and generate several repair models Operation: Fill the repair container layer by layer in the order from skull base to skull top to generate multiple repair model fragments (eg, each layer is 0.5 mm thick).

[0076] Results: Several restoration model fragments were generated, each corresponding to a portion of the temporal bone.

[0077] Step 6: Identify the feature points of each repaired model and the basic 3D model, and merge the repaired model into the basic 3D model according to the similarity of the feature points Operation: Use feature point recognition algorithm to find similar feature points between the repaired model and the base model (such as the outline of the skull, holes, etc.), and then merge the repaired model fragments into the base model one by one.

[0078] Results: The restored skull model is highly consistent with the original model in the temporal bone part and can be used for medical research or surgical planning. Specifically, the steps of selecting the geometric center of the visible hole depth as the repair focus include: Generate the corresponding inner surface based on the visible hole depth; The geometric center of the hole projected on the inner surface is taken as the repair focus.

[0079] By selecting the repair focus, the position of the repair container is fixed, which effectively avoids the problem of repair model displacement caused by the direction and / or position errors of the generated repair model. While effectively reducing the waste of computing power in model repair, it improves the accuracy of repairing models with holes.

[0080] Specifically, the process of filling the repair container at the repair resolution includes: Nesting the repair container on the base 3D model according to the repair resolution; Adjust the angle of the repair container so that any of its faces is parallel to the average plane of the area to be repaired; Adjust the lens position corresponding to the repair resolution to adjust the size of the repair container; The repair container is adjusted so that the projection of any of its faces on the average plane can wrap the projection of the area to be repaired on the average plane.

[0081] Specifically, for a single repair model, the process of selecting its feature points includes: Identify and compare several geometric elements in the repair model and the base model; Overlap the same geometric elements and superimpose the rest of the repaired model with the base model; The geometric elements with a degree of overlap not less than a preset degree of overlap are recorded as feature points of the repaired model.

[0082] Example 4: Based on Example 1, it is assumed that a hole on an antique ceramic vase needs to be repaired, and the corresponding inner surface is generated based on the visible hole depth: Operation: Obtain the depth information of the hole through 3D scanning (assuming the hole depth is 2 cm). According to the shape and depth of the hole, generate a virtual inner surface that simulates the contour inside the hole.

[0083] Result: An inner surface is generated that matches the shape of the hole's interior.

[0084] The geometric center of the hole projected on the inner surface is used as the repair focus: Operation: Project the shape of the hole onto the inner surface and calculate the geometric center of the projection area. This center point is the repair focus.

[0085] Results: The repair focus was determined to be located at the geometric center of the hole at a depth of 1 cm.

[0086] Example 5: Based on Example 2, it is assumed that a tooth tip hole on a mechanical gear needs to be repaired, and the corresponding inner surface is generated based on the visible hole depth.

[0087] Operation: The depth information of the tooth tip hole is obtained through 3D scanning (assuming the depth is 5 mm). According to the shape and depth of the hole, a virtual inner surface is generated, which simulates the contour inside the hole.

[0088] Result: An inner surface is generated that matches the internal shape of the tooth tip hole.

[0089] The geometric center of the hole projected on the inner surface is used as the repair focus: Operation: Project the shape of the hole onto the inner surface and calculate the geometric center of the projection area. This center point is the repair focus.

[0090] Results: The restoration focus was determined to be located at the geometric center of the tooth tip hole at a depth of 2.5 mm.

[0091] Example 6: Based on Example 3, it is assumed that a temporal bone hole on a skull model needs to be repaired, and the corresponding inner surface is generated based on the visible hole depth: Operation: Obtain the depth information of the hole through CT scanning (assuming the depth is 1 cm). According to the shape and depth of the hole, generate a virtual inner surface that simulates the contour of the inside of the hole.

[0092] Result: An inner surface matching the internal shape of the temporal bone foramen was generated.

[0093] The geometric center of the hole projected on the inner surface is used as the repair focus: Operation: Project the shape of the hole onto the inner surface and calculate the geometric center of the projection area. This center point is the repair focus.

[0094] Results: The repair focus was determined to be located at the geometric center of the temporal bone hole at a depth of 0.5 cm.

[0095] Specifically, the pixels corresponding to the repair resolution are not less than twice the target resolution.

[0096] By setting the geometric elements in the model and superimposing the geometric elements, the repair model is superimposed on the basic model, and each model is adjusted according to the display. This effectively improves the precision of the repair of the model with holes and further improves the integrity of the model with holes.

[0097] See also Figure 2 As shown, it is a flow chart of a defect repair method for a three-dimensional reconstruction model of a porous structure according to an embodiment of the present invention, comprising: Step Sq1, generating a basic three-dimensional model of the target object according to the target resolution; Step Sq2, scanning the target object and determining the corresponding area to be repaired; Step Sq3, determining the corresponding repair focus and repair resolution according to the target depth of the area to be repaired; Step Sq4, generating a plurality of repair containers, and placing the geometric centers of the repair containers on the repair focus; Step Sq5, filling the repair container with the repair resolution in a preset order and generating a plurality of repair models; Step Sq6, identifying feature points of each repaired model and the basic three-dimensional model, and merging each repaired model into the basic three-dimensional model; Among them, the repair containers correspond to single feature points respectively.

[0098] Specifically, for a single feature point, a model with a smooth interface transition is selected as the repair model of the feature point; When the repaired container corresponding to each feature point is merged into the basic three-dimensional model, the interface between the corresponding repaired model and the basic three-dimensional model is also smoothed.

[0099] In implementation, it can be specifically composed of the following steps: Step 1: Determine the area of ​​the 3D scanned model of the object with holes that needs to be repaired a. Analyze and determine the missing areas of the 3D scanned model of the object with holes: After obtaining the 3D scanned model of the object with holes, adjust the display angle of the model, observe it from the front, side, top and different angles, and compare it with the original object to find out the missing parts caused by the failure to fully obtain the model information during the 3D scan, and mark the range area. The range area should be marked to spread around the missing part to ensure its integrity during the repair steps later.

[0100] b. Add marks to the hole objects to confirm the size of the area to be repaired: Mark the missing area on the 3D scan model and compare it with the actual object with holes. The area marked on the real object should be greater than or equal to 30% of the area marked on the 3D scan model.

[0101] Step 2: Take a photo of the object with the hole in the area to be repaired a. Place the object with holes in an environment with evenly distributed light, simple background and stable placement of the object (such as a shooting studio), and use a professional digital camera with a pixel of no less than 20 million to shoot the area of ​​the object that needs to be repaired. The area that needs to be repaired should face the camera. When shooting the object, the camera should take a picture every 1 degree horizontally from the vertical 0 degree position around the area that needs to be repaired. After shooting, the camera should be raised vertically by 15 degrees, 30 degrees, 45 degrees, 60 degrees, and 75 degrees with the area that needs to be repaired as the center, and take a picture every 1 degree horizontally.

[0102] b. Photo shooting standards: When shooting, the distance from the center of the object in the area to be repaired to the center of the camera lens should remain unchanged. The size of the area to be repaired in each photo should occupy about 80% of the screen and cannot exceed the screen. The overlap rate of each adjacent photo should be greater than 60%. All photos taken should cover the area inside the hole to be repaired as much as possible from a certain angle to ensure that the key details of the hole are not missed and the morphological characteristics of the hole object at different angles are captured in all directions. The total number of photos taken is related to the size of the area to be repaired, and should be at least 30 or more.

[0103] Step 3: Perform 3D modeling on the photographed photos to generate 3D model A a. Import the completed and qualified photos into the corresponding 3D reconstruction processing system. The system's mature image feature extraction algorithm is used to identify the key feature points in each photo. Based on the multi-view geometry principle in the field of computer vision and combined with the triangulation algorithm, point cloud data is generated based on these feature points and the corresponding camera position information.

[0104] b. According to the spatial distribution law of point cloud data and the topological relationship between adjacent points, the discrete point cloud data is converted into continuous mesh data through surface fitting operation. The mesh data forms the surface model of the object in the form of triangular facets or quadrilateral facets, presenting the three-dimensional shape of the area to be repaired of the object with holes and generating a three-dimensional model A.

[0105] Step 4: Scale and align the size and position information of the 3D model A to be the same as that of the 3D scanned model. Place the 3D scanned model and the 3D model A in the same 3D spatial reference system for processing. By comparing the overall sizes of the two, calculate the scale factor of the size of the 3D model A to the 3D scanned model. According to the scale factor, use the corresponding scaling algorithm to scale the 3D model A so that it is basically consistent with the 3D scanned model in size. Then, using the principle of spatial geometric position transformation and the surface similar feature point alignment method, perform translation, rotation and other operations on the 3D model A, and continuously adjust its spatial position, so that the relative positions of the 3D model A and the 3D scanned model in the 3D space gradually become consistent. During the alignment and adjustment process, continuously evaluate the overlap between the two, repeatedly optimize and adjust the X, Y, and Z axis values ​​of the 3D model A and its overall volume size, until the overlap between the two can be close to 95% or more, minimize the error as much as possible, and ensure that the axis size error of the three coordinates (X, Y, and Z axes) in the 3D space is less than or equal to 0.001mm.

[0106] See also Figure 3 As shown, it is a schematic diagram of three-dimensional model alignment according to an embodiment of the present invention; See also Figure 3 Figure (a) is a schematic diagram of the misalignment of the three-dimensional model of an embodiment of the present invention; In the figure, the repaired model and the corresponding target model are in a misaligned state; See also Figure 3 Figure (b) is a schematic diagram of alignment of a three-dimensional model according to an embodiment of the present invention; In the figure, the repaired model and the corresponding target model are in alignment; Step 5: Merge 3D model A and 3D scan model to generate 3D model B Import the two models completed in step 4: 3D model A and 3D scan model into the 3D model processing software. Compare and fuse the corresponding geometric elements in the two models, such as faces and vertices, one by one. For the geometric elements of the overlapping parts, integrate them according to reasonable weight distribution or take the optimal value to ensure that there will be no structural conflicts or data loss during the merging process. After the above merging operation, the 3D model A and the 3D scan model are merged into a complete 3D model B, which contains the 3D structural information of the hole-related parts of the missing parts of the 3D scan model, forming a more comprehensive and accurate 3D model B.

[0107] Step 6: Repair the overlapping parts of the 3D model B. For the generated 3D model B, observe the overlapping parts of the model surface. According to the geometric features of the facets and their positional relationship in the overall structure of the model, selectively delete or reasonably merge and adjust the overlapping faces to remove the abnormal meshes generated by the repair. During the processing, it is necessary to ensure that the overall structural integrity and geometric accuracy of the model are not affected. After such repair operations, the quality of the 3D model B is improved.

[0108] See also Figure 4 As shown, it is a schematic diagram of a model with holes repaired according to an embodiment of the present invention; See also Figure 4 Figure (a) is a three-dimensional model with a heavy surface before restoration according to an embodiment of the present invention; It is not difficult to see that the location with the hole is composed of several overlapping faces; See also Figure 4 Figure (b) is a three-dimensional model without a heavy surface after restoration according to an embodiment of the present invention; Principles and advantages of image modeling technology in this application: principle: Use a high-pixel professional digital camera to shoot the object from multiple different angles. During the shooting process, although reflection and refraction may occur after the light enters the hole, shooting from different angles can make these light rays shoot out of the hole from different paths and be captured by the camera. Then, the multi-angle photos taken are processed by computer vision algorithms. First, image preprocessing is performed, including image denoising, contrast enhancement and other operations to improve image quality. Then feature extraction is performed to identify the contours, edges and other feature information of the hole in different images. After that, the three-dimensional reconstruction algorithm is used to calculate the coordinates of the feature points in the three-dimensional space according to the correspondence between the feature points in the image at different angles, combined with the camera's shooting parameters (such as focal length, shooting position, etc.), and the point cloud data of the hole is gradually constructed to generate a three-dimensional model.

[0109] Unify the X, Y, and Z axis coordinate parameters of the two models, and scale and align the two different 3D models to fixed coordinates in the same 3D space to make them overlap based on the matching method of similar feature points of the two 3D models. This can accurately repair the holes in the objects and provide a basis for subsequent repair work.

[0110] Advantages: 1. High precision: Through multi-angle shooting, the detailed features of the hole at different angles can be captured, avoiding the omission of information caused by single-angle scanning. For example, for a hole with complex textures inside, multi-angle shooting can obtain texture information from all directions, so that these details can be restored more accurately when using photos for modeling, which is significantly improved in accuracy compared to existing 3D scanning technology. This improvement in accuracy is mainly due to the use of high-pixel professional digital cameras to collect all-round image information of the object's holes, which makes up for the defect of losing model surface information due to angle limitations when using 3D scanning technology.

[0111] 2. Completeness: It is possible to obtain information from deep holes, micro holes or holes with branched structures. By combining appropriate shooting angles, it is possible to obtain relatively complete internal structural information of the hole of an object. Compared with using only 3D scanning technology, there will be no large amount of information loss due to reflection, refraction or occlusion of the scanning light in the hole. This is due to the fact that multi-angle shooting can flexibly adjust the shooting position and angle, thus overcoming the limitations of the fixed light propagation path in the use of 3D scanning technology.

[0112] 3. High flexibility: no complex scanning equipment and specific scanning environment are required. High-pixel professional digital cameras can be used for multi-angle shooting, and the shooting plan can be flexibly adjusted according to the shape, size and hole position of the object. However, 3D scanning technology often relies on large, expensive and fixed scanning equipment, and has high requirements for the scanning environment (such as light, space, etc.).

[0113] In addition, as an alternative solution, when dealing with the problem of obtaining and repairing the information of holes inside objects, if multi-angle shooting image modeling and three-dimensional reconstruction technology are not used, it can be reproduced and restored with long-term carving experience and professional software. Workers need long-term training, in-depth understanding of material and structural characteristics, and accumulation of superb skills. With the real object as a reference, first accurately measure and enter the feature data to build a virtual model, and then manipulate the tools based on experience and skills to reproduce the carving. Although this solution can achieve hole restoration, it relies on long-term experience accumulation, is time-consuming, and is limited by personal skills and talents. It is difficult to promote widely, and its accuracy and efficiency are not as good as the current technical method of the present invention.

[0114] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

[0115] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for repairing defects in a three-dimensional reconstruction model of a porous structure, characterized in that: include: Generate a basic 3D model of the target object at a target resolution; Scan the target object and determine the corresponding area to be repaired; Determine the corresponding repair focus and repair resolution according to the target depth of the area to be repaired; Generate a repair container, and place the geometric center of the repair container on the repair focus; Filling the repair container with the repair resolution in a preset order and generating a plurality of repair models; Identify feature points of each repair model and the basic three-dimensional model, and merge the repair model into the basic three-dimensional model according to the similarity of the feature points; Wherein, the repair container wraps at least one characteristic point.

2. The defect repair method for a three-dimensional reconstruction model of a porous structure according to claim 1, characterized in that: Steps to identify areas for restoration include: Pre-segmenting the basic three-dimensional model based on the size of the repair container to form a plurality of segmentation lines; By comparing the basic three-dimensional model, determining the position of each hole in the three-dimensional model; Adjusting the segmentation line to adjust the position of each hole to the complete area of ​​the basic three-dimensional model; The basic three-dimensional model is segmented according to the segmentation line to form a corresponding area to be repaired.

3. The defect repair method for a three-dimensional reconstruction model of a porous structure according to claim 1, characterized in that: The steps of generating the repair focus include: determining a maximum hole depth in the repair container; Determining the visible hole depth according to the maximum hole depth; Based on the visible hole depth, the geometric center of the visible hole depth is selected as the repair focus.

4. The defect repair method for a three-dimensional reconstruction model of a porous structure according to claim 3, characterized in that: The step of determining the visible hole depth comprises: determining a container outer surface of the repaired container and a model outer surface of the basic three-dimensional model contained in the repaired container; Selecting holes on the outer surface of the model, and taking the geometric center of the repaired container as a reference, projecting the holes on the outer surface of the container to form a shooting reference plane; The hole corresponding to the shooting reference plane is photographed at the target resolution, and the depth that can be resolved at the target resolution is recorded as the visible hole depth.

5. The defect repair method for a three-dimensional reconstruction model of a porous structure according to claim 4, characterized in that: The steps of selecting the geometric center of the visible hole depth as the repair focus include: Generate a corresponding inner surface based on the visible hole depth; The geometric center of the hole projected on the inner surface is used as the repair focus.

6. The defect repair method for a three-dimensional reconstruction model of a porous structure according to claim 2 or 5, characterized in that: The process of filling the repair container at the repair resolution includes: According to the repair resolution, nesting the repair container on the basic three-dimensional model; Adjusting the angle of the repair container so that any one of its faces is parallel to the average plane of the area to be repaired; Adjusting the lens position corresponding to the restoration resolution to adjust the size of the restoration container; The repair container is adjusted so that the projection of any surface of the repair container on the average plane can wrap the projection of the area to be repaired on the average plane.

7. The defect repair method for a three-dimensional reconstruction model of a porous structure according to claim 6, characterized in that: For a single repair model, the process of selecting its feature points includes: Identify and compare several geometric elements in the repair model and the base model; Superimpose the same geometric elements, and superimpose the remaining positions of the repaired model with the base model; The geometric elements whose overlapping degree is not less than a preset overlapping degree are recorded as feature points of the repair model.

8. The defect repair method for a three-dimensional reconstruction model of a porous structure according to claim 7, characterized in that: The pixels corresponding to the repair resolution are not less than 2 times of the target resolution.

9. A method for repairing defects in a three-dimensional reconstruction model of a porous structure, characterized in that: include: Generate a basic 3D model of the target object at a target resolution; Scan the target object and determine the corresponding area to be repaired; Determine the corresponding repair focus and repair resolution according to the target depth of the area to be repaired; Generate a plurality of repair containers, and place the geometric centers of the repair containers on the repair focus; Filling the repair container with the repair resolution in a preset order and generating a plurality of repair models; Identify feature points of each repair model and the basic three-dimensional model, and merge each repair model into the basic three-dimensional model; Wherein, the repair containers respectively correspond to single feature points.

10. The defect repair method for a three-dimensional reconstruction model of a porous structure according to claim 9, characterized in that: For a single feature point, a model with smooth interface transition is selected as the repair model of the feature point; When the repaired container corresponding to each feature point is merged into the basic three-dimensional model, the interface between the corresponding repaired model and the basic three-dimensional model is also smoothed.

Images