Intelligent interaction method and system for ancient building restoration based on image feature recognition

Abstract

The application provides an ancient building repair intelligent interaction method and system based on image feature recognition, wherein the method comprises the following steps: acquiring multi-angle images of a target ancient building needing repair; extracting image features of the multi-angle images and determining repair content; performing three-dimensional reconstruction based on the multi-angle images, acquiring a three-dimensional reconstruction model, and labeling the repair content in the three-dimensional reconstruction model; acquiring a repair progress of a repair personnel; and automatically adjusting a model angle of the three-dimensional reconstruction ancient building model according to the repair progress for the repair personnel to view. The ancient building repair intelligent interaction method and system based on image feature recognition extract image features of multi-angle images of a target ancient building, determine repair content, perform three-dimensional reconstruction on the multi-angle images, label the repair content in the three-dimensional reconstruction model, acquire an ancient building model, automatically synchronously adjust a model angle of the ancient building model based on the acquired repair progress of a repair personnel, and enable the repair personnel to view, which is more convenient and intuitive.

Description

Technical Field

[0001] This invention relates to the field of ancient building restoration technology, and in particular to an intelligent interactive method and system for ancient building restoration based on image feature recognition. Background Technology

[0002] As carriers of historical information and cultural memory, ancient buildings reflect the architectural art, technology, and cultural level of their time. As irreplaceable cultural heritage and physical evidence of history, their maintenance and management are of great significance. Ancient building restoration is an important cultural and historical protection activity. Currently, ancient building restoration mainly relies on experienced restorers who perform restoration and maintenance based on their experience. However, these restorers need to find relevant historical materials as a basis for restoration and cannot obtain direct guidance, making the process quite cumbersome.

[0003] In view of this, there is an urgent need for intelligent interactive methods and systems for the restoration of ancient buildings based on image feature recognition, in order to at least address the above-mentioned shortcomings. Summary of the Invention

[0004] One of the objectives of this invention is to provide an intelligent interactive method for the restoration of ancient buildings based on image feature recognition. This method extracts image features from multi-angle images of the target ancient building, determines the restoration content, performs three-dimensional reconstruction of the multi-angle images, marks the restoration content in the three-dimensional reconstruction model, obtains the ancient building model, and automatically adjusts the model angle of the ancient building model in sync with the restoration progress of the restoration personnel for their viewing. This eliminates the need for manual switching of the ancient building model's viewing angle, making it more convenient and intuitive.

[0005] The intelligent interactive method for ancient building restoration based on image feature recognition provided in this invention includes:

[0006] Step 1: Obtain multi-angle images of the target ancient building that needs to be restored;

[0007] Step 2: Extract image features from multi-angle images and determine the content to be repaired;

[0008] Step 3: Perform 3D reconstruction based on multi-angle images to obtain a 3D reconstruction model. Mark the repair content in the 3D reconstruction model to obtain the ancient building model.

[0009] Step 4: Obtain the repair progress from the repair personnel;

[0010] Step 5: The model angle of the 3D reconstructed ancient building model is automatically adjusted according to the restoration progress for the restoration personnel to view.

[0011] Preferably, step 1: Obtain multi-angle images of the target ancient building that needs to be restored, including:

[0012] Obtain the airspace range of the target ancient building;

[0013] Release the drone within the airspace based on its flight altitude and initial flight angle;

[0014] Obtain the scan results from the drone;

[0015] Based on the preset elevation feature extraction template, the elevation features are determined according to the scanning results;

[0016] Based on the preset elevation difference correlation feature extraction template, the elevation difference correlation features are determined according to the scanning results and elevation features, and the combination relationship between elevation features and elevation difference correlation features is established.

[0017] Obtain the shooting angle range of the drone and extract the shooting angle range features;

[0018] Based on the combination relationship and the shooting angle range characteristics, determine the second flight angle of the UAV in the local airspace range of the elevation characteristics corresponding to the combination relationship;

[0019] When the drone reaches the local airspace, control the drone to fly at the second flight angle and take pictures;

[0020] Once all local airspace areas have been photographed, multi-angle images are obtained.

[0021] Preferably, based on the combination relationship and the shooting angle range characteristics, the second flight angle of the UAV within the local airspace range corresponding to the elevation characteristics of the combination relationship is determined, including:

[0022] Based on the combination relationship, a local elevation model is constructed for the local spatial range of the elevation features corresponding to the combination relationship;

[0023] Identify the UAV model within the local elevation model;

[0024] Based on the model position and shooting angle range characteristics of the UAV model, the simulated shooting field of view area in the local elevation model is determined;

[0025] The basis for extracting elevation difference correlation features corresponding to the combination relationships in the local elevation model is determined by the location model.

[0026] Identify the reference part model that conforms to the characteristics of the standard reference part model, and use it as the standard reference part model;

[0027] Obtain the standard reference model corresponding to the flight angle adjustment reference point determination rule;

[0028] Based on the rules for determining the reference points for flight angle adjustment, determine the reference points for flight angle adjustment in the local elevation model;

[0029] The second flight angle is determined by adjusting the reference point and the simulated area of ​​the shooting field of view based on the flight angle adjustment.

[0030] Preferably, the second flight angle is determined based on the adjusted reference point and the simulated shooting field of view, including:

[0031] Determine whether the point for adjusting the flight angle is within the simulated shooting field of view;

[0032] If not, the corresponding flight angle adjustment reference point will be used as the target point;

[0033] Connect the target point to the model position of the UAV model by a target line;

[0034] Determine the boundaries of the two sides of the simulated field of view, and take the boundary of the side of the two sides that is relatively closer to the line connecting the target as the target boundary;

[0035] Calculate the angle between the target line and the target boundary, and determine the maximum angle corresponding to the local elevation model;

[0036] The second flight angle is the flight angle based on the maximum angle between the drone's velocity direction and the vertical upward tilt of the horizontal plane.

[0037] Preferably, step 2: extract image features from multi-angle images and determine the content to be repaired, including:

[0038] Determine standard image features based on multi-angle images;

[0039] Image features are matched with standard image features to identify local images where the image features do not match.

[0040] The restoration content is determined based on the image features corresponding to the local images and the pre-set ancient architecture knowledge graph database.

[0041] Preferably, step 3: Perform 3D reconstruction based on multi-angle images to obtain a 3D reconstruction model, annotate the repair content in the 3D reconstruction model, and obtain an ancient building model, including:

[0042] Based on the content to be repaired, identify the missing model;

[0043] Based on the pre-defined annotation rules for the missing model, the missing model is annotated in the 3D reconstruction model;

[0044] Determine the repair strategy based on the content to be repaired;

[0045] The repair strategy is displayed in the highlighted area, where there is no overlap between the highlighted area and the 3D reconstructed model and the missing model.

[0046] Preferably, step 4: Obtain the repair progress from the repair personnel, including:

[0047] Based on the content to be repaired, obtain the standard repair process;

[0048] Acquire images of the restoration process by the restorers;

[0049] The repair process is determined based on the repair process images and the standard repair process.

[0050] Preferably, step 5: The angle of the 3D reconstructed ancient building model is automatically adjusted according to the restoration progress for restoration personnel to view, including:

[0051] Get the current viewing angle of the model;

[0052] Determine the ideal viewing angle for the ancient building model based on the restoration progress;

[0053] Determine if the current model viewing angle is consistent with the ideal model viewing angle. If they are inconsistent, adjust the current model viewing angle to the ideal model viewing angle for the repair personnel to view.

[0054] Preferably, based on the restoration progress, the ideal viewing angle for the ancient building model is determined, including:

[0055] Determine the current scope of repair based on the repair progress;

[0056] Determine the current restoration area as a partial model corresponding to the ancient building model;

[0057] Obtain the angular relationship between the local model and the ancient building model;

[0058] Determine the ideal viewing angle for the model based on the angular relationships.

[0059] The intelligent interactive system for ancient building restoration based on image feature recognition provided in this invention includes:

[0060] The ancient building image acquisition subsystem is used to acquire multi-angle images of the target ancient building that needs to be restored;

[0061] The repair content determination subsystem is used to extract image features from multi-angle images and determine the repair content;

[0062] The repair content annotation subsystem is used for 3D reconstruction based on multi-angle images, to obtain a 3D reconstruction model, to annotate the repair content in the 3D reconstruction model, and to obtain an ancient building model.

[0063] The repair process acquisition subsystem is used to acquire the repair process of repair personnel;

[0064] The intelligent interactive subsystem is used to automatically adjust the angle of the 3D reconstructed ancient building model according to the restoration process, so that the restoration personnel can view it.

[0065] The beneficial effects of this invention are as follows:

[0066] This invention extracts image features from multi-angle images of the target ancient building, determines the restoration content, performs three-dimensional reconstruction of the multi-angle images, marks the restoration content in the three-dimensional reconstruction model, obtains the ancient building model, and automatically adjusts the model angle of the ancient building model in sync with the restoration progress of the restoration personnel for the restoration personnel to view, without the need for manual switching of the ancient building model observation angle, which is more convenient and intuitive.

[0067] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in this application.

[0068] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0069] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0070] Figure 1 This is a schematic diagram of the intelligent interactive method for ancient building restoration based on image feature recognition in an embodiment of the present invention;

[0071] Figure 2 This is a schematic diagram of an intelligent interactive system for the restoration of ancient buildings based on image feature recognition, as described in an embodiment of the present invention. Detailed Implementation

[0072] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0073] This invention provides an intelligent interactive method for the restoration of ancient buildings based on image feature recognition, such as... Figure 1 As shown, it includes:

[0074] Step 1: Obtain multi-angle images of the target ancient building that needs to be restored; where the target ancient building is the ancient building that needs to be restored, such as a group of ancient building towers; the multi-angle images are: images of the target ancient building taken from different angles by a drone;

[0075] Step 2: Extract image features from multi-angle images and determine the repair content; where image features are characteristic representations of the captured images of ancient buildings, such as the edges, corners, textures, and shapes of the ancient buildings in the images; the repair content is: the ancient building areas that need to be repaired and the repair methods.

[0076] Step 3: Perform 3D reconstruction based on multi-angle images to obtain a 3D reconstruction model. Annotate the repair content in the 3D reconstruction model to obtain the ancient building model. The 3D reconstruction model is a 3D model generated by fusing 2D multi-angle images. When annotating, add reminder information to assist in the repair.

[0077] Step 4: Obtain the repair progress of the repair personnel; where repair progress refers to the current repair progress of the repair personnel, specifically which stage of the repair they are in, such as: repair of the wooden structure inside the tower, or: reinforcement of the tower body;

[0078] Step 5: The model angle of the 3D reconstructed ancient building model is automatically adjusted according to the restoration progress for the restoration personnel to view. The model angle refers to the observation angle of the ancient building model; specifically, the area of ​​the ancient building model observed by the automatically adjusted observation angle corresponds to the area of ​​the ancient building in the real-time restoration process.

[0079] The working principle and beneficial effects of the above technical solution are as follows:

[0080] This invention extracts image features from multi-angle images of the target ancient building, determines the restoration content, performs three-dimensional reconstruction of the multi-angle images, marks the restoration content in the three-dimensional reconstruction model, obtains the ancient building model, and automatically adjusts the model angle of the ancient building model in sync with the restoration progress of the restoration personnel for the restoration personnel to view, without the need for manual switching of the ancient building model observation angle, which is more convenient and intuitive.

[0081] In one embodiment, step 1: acquiring multi-angle images of the target ancient building that needs restoration, including:

[0082] Obtain the airspace range of the target ancient building; whereby the airspace range is the space above the target ancient building that is available for drone flight;

[0083] The drone is released within the airspace based on its flight altitude and initial flight angle; the flight altitude is greater than the highest height of the ancient building; the flight angle is the angle between the line containing the drone's velocity direction and the horizontal plane, and the initial flight angle is 0 degrees.

[0084] Acquire the scanning results of the drone; wherein, the scanning results are: the laser scanning results of the laser radar pre-positioned below the drone in the direction perpendicular to the ground;

[0085] Based on a preset elevation feature extraction template, elevation features are determined according to the scanning results. The elevation feature extraction template is used to extract elevation information by referring to the scanning results. For example, it is a template for parsing the scanning results to obtain height values ​​and coordinate positions. The elevation feature is: what is the height at what location.

[0086] Based on a preset elevation difference correlation feature extraction template, elevation difference correlation features are determined according to the scanning results and elevation features, and a combination relationship between elevation features and elevation difference correlation features is established. The elevation difference correlation feature extraction template is a difference relationship extraction template when there are differences in the elevation features of adjacent coordinates. The elevation difference correlation features are: the boundary coordinates of adjacent coordinate regions with elevation differences and the coordinate regions corresponding to the elevation features, and the elevation difference value. For example, if the elevation value of coordinate region A is 10 meters, and the elevation value of the adjacent coordinate region B is 15 meters, then the corresponding elevation difference correlation features are: the boundary coordinates of coordinate regions A and B, and -5 meters.

[0087] Obtain the shooting angle range of the drone and extract the shooting angle range features; where the shooting angle range is: the shooting angle range of the camera mounted on the drone; and the shooting angle range features are: the shooting angle values ​​of the drone.

[0088] Based on the combination relationship and the shooting angle range characteristics, the second flight angle of the UAV in the local airspace range of the elevation characteristics corresponding to the combination relationship is determined; wherein, the second flight angle is: the shooting angle of the UAV determined based on the elevation difference correlation characteristics;

[0089] When the drone reaches the local airspace, control the drone to fly at the second flight angle and take pictures;

[0090] Once all local spatial domains have been photographed, multi-angle images are obtained;

[0091] Specifically, based on the combination relationship and the shooting angle range characteristics, the second flight angle of the UAV within the local airspace range corresponding to the elevation characteristics of the combination relationship is determined, including:

[0092] Based on the combination relationship, a local elevation model is constructed for the local spatial range of the elevation features corresponding to the combination relationship; wherein, the local elevation model is: a local ancient building model constructed based on the elevation features and the elevation difference correlation features corresponding to the elevation features;

[0093] Identify the UAV model within the local elevation model;

[0094] Based on the model position and shooting angle range characteristics of the UAV model, the shooting field of view simulation area in the local elevation model is determined; wherein, the model position of the UAV model is determined according to the elevation characteristics; the simulated shooting points of the shooting field of view simulation area are the points of the virtual camera on the UAV model, and the area boundary of the shooting field of view simulation area is determined according to the shooting angle range characteristics.

[0095] The extraction basis model for the elevation difference correlation features corresponding to the combination relationship in the local elevation model is determined; wherein, the extraction basis model is the boundary between the simulated area corresponding to the elevation feature in the local elevation model and the simulated area where there is an elevation difference.

[0096] Identify the reference location models that conform to the characteristics of the standard reference location model and use them as the standard reference location models; wherein, the characteristics of the standard reference location model are: reference location models that are higher than the local elevation model corresponding to the elevation characteristics.

[0097] Obtain the standard reference model corresponding to the flight angle adjustment reference point determination rules; among which, the flight angle adjustment reference point determination rules are: the rules for calculating the coordinates of the simulated area boundary based on the simulated area boundary;

[0098] Based on the rules for determining the reference points for flight angle adjustment, the reference points for flight angle adjustment in the local elevation model are determined; among them, the reference points for flight angle adjustment are: the coordinates of the boundary of the simulated area;

[0099] Based on the flight angle adjustment reference point and the simulated shooting field of view area, determine the second flight angle;

[0100] The second flight angle is determined by adjusting the reference point and the simulated area of ​​the shooting field of view based on the flight angle adjustment, including:

[0101] Determine whether the point for adjusting the flight angle is within the simulated shooting field of view;

[0102] If not, the corresponding flight angle adjustment reference point will be used as the target point;

[0103] Connect the target point to the model position of the UAV model by a target line;

[0104] Determine the boundaries of the two sides of the simulated field of view, and take the boundary of the side of the two sides that is relatively closer to the line connecting the target as the target boundary;

[0105] Calculate the angle between the target line and the target boundary, and determine the maximum angle corresponding to the local elevation model;

[0106] The second flight angle is the flight angle based on the maximum angle between the drone's velocity direction and the vertical upward tilt of the horizontal plane.

[0107] The working principle and beneficial effects of the above technical solution are as follows:

[0108] This invention controls a drone to enter and capture multi-angle images within the airspace of a target ancient building. It introduces an elevation feature extraction template to extract the elevation features of the target ancient building area. Since there are obstruction areas between buildings of varying heights, and the drone's shooting range in one direction is limited, an elevation difference correlation feature extraction template is introduced. This template determines the elevation difference correlation features (the boundary coordinates and elevation difference values ​​of adjacent coordinate regions with elevation differences and the coordinate regions corresponding to the elevation features), establishing a combination relationship between the elevation features and the elevation difference correlation features. Simultaneously, it extracts the shooting angle range features of the drone's shooting angle range. Based on the combination relationship and the shooting angle range features, it determines the second flight angle of the local airspace range corresponding to the elevation features of the combination relationship. When the drone reaches the local airspace range, it is controlled to fly and capture images based on the second flight angle. After capturing images of all local airspace ranges, multi-angle images are obtained, improving the comprehensiveness of the multi-angle images and enhancing the accuracy of subsequent 3D reconstruction model construction.

[0109] Specifically, when determining the second flight angle of the UAV within the local airspace range of the elevation features corresponding to the combination relationship based on the combination relationship and the shooting angle range characteristics, a local elevation model of the local airspace range of the elevation features corresponding to the combination relationship is constructed. The UAV model is determined based on the UAV's three-dimensional parameters, the model position of the UAV model is determined based on the elevation features, and the simulated shooting field of view area in the local elevation model is determined based on the model position and the shooting angle range characteristics. Based on the elevation difference correlation features corresponding to the combination relationship in the local elevation model, an extraction basis part model is constructed, a standard basis part model with an altitude above the elevation features is determined, and based on the relative positional relationship between the standard basis part model and the local elevation model, the rules for determining the flight angle adjustment basis point are determined. Based on the rules for determining the flight angle adjustment basis point, the flight angle adjustment basis point in the local elevation model is determined.

[0110] To determine whether the flight angle adjustment reference point is within the simulated field of view, if it is not, it means that the image of the ancient building side wall corresponding to the model of the corresponding standard reference part cannot be fully captured, which will lead to inaccurate 3D reconstruction. Therefore, the target line connecting the corresponding flight angle adjustment reference point and the model position is obtained. At the same time, the boundary of the area relatively closer to the target line in the boundary of the two sides of the simulated field of view is determined as the target boundary. The angle between the target line and the target boundary is calculated. Since the local elevation model may correspond to multiple target points, the largest angle is selected to ensure that all high-level building areas can be captured. The flight angle with the maximum angle of vertical upward tilt of the UAV speed direction based on the horizontal plane is used as the second flight angle, which improves the suitability of the second flight angle determination process, further improves the richness of 3D reconstruction materials, and enhances the reliability of the restoration basis.

[0111] In one embodiment, step 2: extracting image features from multi-angle images and determining the content to be repaired includes:

[0112] Based on multi-angle images, standard image features are determined; among them, standard image features are: characteristic representations of standard target ancient buildings, extracted from recorded information of the target ancient buildings before they were damaged (such as: historical images, historical descriptions);

[0113] Image features are matched with standard image features to identify local images where the image features do not match; where local images are local regions of multi-angle images corresponding to the mismatched features between the image features and the standard image features.

[0114] The restoration content is determined based on the image features corresponding to the local image and the preset ancient architecture knowledge graph database. The preset ancient architecture knowledge graph database is a database containing ancient architecture knowledge, such as architectural style, architectural techniques, and restoration methods.

[0115] The working principle and beneficial effects of the above technical solution are as follows:

[0116] This invention introduces standard image features to match the image features of multi-angle images, identifies local images where the image features do not match, and determines the restoration content based on the image features corresponding to the local images and the introduced ancient architecture knowledge graph database, making the determination of restoration content more reasonable.

[0117] In one embodiment, step 3: Perform 3D reconstruction based on multi-angle images to obtain a 3D reconstruction model, annotate the repair content in the 3D reconstruction model, and obtain an ancient building model, including:

[0118] Based on the content to be repaired, the missing model is determined; whereby the missing model is: the three-dimensional model of the area that needs to be repaired;

[0119] Based on the pre-defined annotation rules of the missing model, the missing model is annotated in the 3D reconstruction model; the pre-defined annotation rules are set manually in advance, for example: dotted lines are used to annotate the missing positions of the ancient building model.

[0120] Based on the content of the repair, determine the repair strategy; the repair strategy includes: repair plan, including: what repair materials to use, what repair techniques to use, and precautions, etc.

[0121] The repair strategy is displayed in the highlighted area, where there is no overlap between the highlighted area and the 3D reconstructed model and the missing model.

[0122] The working principle and beneficial effects of the above technical solution are as follows:

[0123] This invention constructs a missing model of the missing part based on the content to be repaired, annotates the missing model in the 3D reconstruction model, and displays the text information of the repair strategy in the reminder box area next to the 3D reconstruction model. It also displays the textual repair strategy and the graphical result of the repair, thus improving the comprehensiveness of the intelligent repair interaction.

[0124] In one embodiment, step 4: obtaining the repair personnel's repair progress includes:

[0125] Based on the content to be repaired, obtain the standard repair process; where the standard repair process is the result obtained by standardizing the repair process according to the repair strategy in the content to be repaired.

[0126] Acquire images of the restoration process by the restoration personnel; wherein, the restoration process images are: images recording the process of the restoration personnel carrying out the restoration of the ancient building, acquired from the on-site monitoring device.

[0127] The repair process is determined based on the repair process images and the standard repair process.

[0128] The working principle and beneficial effects of the above technical solution are as follows:

[0129] This invention determines the standard repair process corresponding to the repair strategy, and determines the sub-process corresponding to the standard repair process based on the repair process image, and uses it as the repair process. The determination of the repair process is more reasonable, and further improves the accuracy of the adjustment basis for subsequent angle adjustments.

[0130] In one embodiment, step 5: automatically adjusting the angle of the 3D reconstructed ancient building model according to the restoration progress for restoration personnel to view, including:

[0131] Get the current model viewing angle; where the current model viewing angle is: the model viewing angle of the real-time display of the model screen on the ancient building model viewing terminal interface relative to the ancient building model, and the model viewing angle is: different perspectives selected by rotating the view around the ancient building model;

[0132] Based on the restoration progress, determine the current restoration scope; where the current restoration scope is: the area of ​​the ancient building model that needs to be restored corresponding to the restoration progress.

[0133] The current restoration area corresponds to a local model of the ancient building model; where the local model is the part of the local model that contains the smallest part of the ancient building model with the current restoration area.

[0134] Obtain the angular relationship between the local model and the ancient building model; where the angular relationship refers to the spatial orientation relationship between the local model and the ancient building model.

[0135] Based on the angular relationship, determine the ideal viewing angle of the model; whereby the ideal viewing angle of the model is: the best view angle of the ancient building model that allows observation of the local model.

[0136] Determine if the current model viewing angle is consistent with the ideal model viewing angle. If they are inconsistent, adjust the current model viewing angle to the ideal model viewing angle for the repair personnel to view.

[0137] The working principle and beneficial effects of the above technical solution are as follows:

[0138] This invention determines the local model of the current restoration scope in the restoration process, and then determines the best observation angle for restoration based on the ancient building model, i.e., the ideal model viewing angle, according to the angular relationship between the local model and the ancient building model. It also determines whether the current model viewing angle and the ideal model viewing angle are consistent. If they are inconsistent, the system automatically adjusts the current model viewing angle to the ideal model viewing angle for the restoration personnel to view, eliminating the need for manual view switching and making it more convenient.

[0139] This invention provides an intelligent interactive system for the restoration of ancient buildings based on image feature recognition, such as... Figure 2 As shown, it includes:

[0140] Ancient building image acquisition subsystem 1 is used to acquire multi-angle images of the target ancient building that needs to be restored;

[0141] Repair content determination subsystem 2 is used to extract image features from multi-angle images and determine the repair content;

[0142] The repair content annotation subsystem 3 is used to perform 3D reconstruction based on multi-angle images, obtain a 3D reconstruction model, annotate the repair content in the 3D reconstruction model, and obtain an ancient building model.

[0143] Repair process acquisition subsystem 4 is used to acquire the repair process of repair personnel;

[0144] The intelligent interactive subsystem 5 is used to automatically adjust the angle of the 3D reconstructed ancient building model according to the restoration process for the restoration personnel to view.

[0145] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. An intelligent interactive method for the restoration of ancient buildings based on image feature recognition, characterized in that, include: Step 1: Obtain the airspace range of the target ancient building; Release the drone within the airspace based on its flight altitude and initial flight angle; Obtain the scan results from the drone; Based on the preset elevation feature extraction template, the elevation features are determined according to the scanning results; Based on the preset elevation difference correlation feature extraction template, the elevation difference correlation features are determined according to the scanning results and elevation features, and the combination relationship between elevation features and elevation difference correlation features is established. Obtain the shooting angle range of the drone and extract the shooting angle range features; Based on the combination relationship, a local elevation model is constructed for the local spatial range of the elevation features corresponding to the combination relationship; Identify the UAV model within the local elevation model; Based on the model position and shooting angle range characteristics of the UAV model, the simulated shooting field of view area in the local elevation model is determined; The basis for extracting elevation difference correlation features corresponding to the combination relationships in the local elevation model is determined by the location model. Identify the reference part model that conforms to the characteristics of the standard reference part model, and use it as the standard reference part model; Obtain the standard reference model corresponding to the flight angle adjustment reference point determination rule; Based on the rules for determining the reference points for flight angle adjustment, determine the reference points for flight angle adjustment in the local elevation model; Determine whether the point for adjusting the flight angle is within the simulated shooting field of view; If not, the corresponding flight angle adjustment reference point will be used as the target point; Connect the target point to the model position of the UAV model by a target line; Determine the boundaries of the two sides of the simulated field of view, and take the boundary of the side of the two sides that is relatively closer to the line connecting the target as the target boundary; Calculate the angle between the target line and the target boundary, and determine the maximum angle corresponding to the local elevation model; The second flight angle is the flight angle based on the maximum angle between the drone's velocity direction and the vertical upward tilt of the horizontal plane. When the drone reaches the local airspace, control the drone to fly at the second flight angle and take pictures; Once all local spatial domains have been photographed, multi-angle images are obtained; Step 2: Extract image features from multi-angle images and determine the content to be repaired; Step 3: Perform 3D reconstruction based on multi-angle images to obtain a 3D reconstruction model. Mark the repair content in the 3D reconstruction model to obtain the ancient building model. Step 4: Obtain the repair progress from the repair personnel; Step 5: The model angle of the 3D reconstructed ancient building model is automatically adjusted according to the restoration progress for the restoration personnel to view.

2. The intelligent interactive method for ancient building restoration based on image feature recognition as described in claim 1, characterized in that, Step 2: Extract image features from multi-angle images and determine the content to be repaired, including: Determine standard image features based on multi-angle images; Image features are matched with standard image features to identify local images where the image features do not match. The restoration content is determined based on the image features corresponding to the local images and the pre-set ancient architecture knowledge graph database.

3. The intelligent interactive method for ancient building restoration based on image feature recognition as described in claim 1, characterized in that, Step 3: Perform 3D reconstruction based on multi-angle images to obtain a 3D reconstructed model. Annotate the repaired content on the 3D reconstructed model to obtain the ancient building model, including: Based on the content to be repaired, identify the missing model; Based on the pre-defined annotation rules for the missing model, the missing model is annotated in the 3D reconstruction model; Determine the repair strategy based on the content to be repaired; The repair strategy is displayed in the highlighted area, where there is no overlap between the highlighted area and the 3D reconstructed model and the missing model.

4. The intelligent interactive method for ancient building restoration based on image feature recognition as described in claim 1, characterized in that, Step 4: Obtain the repair progress from the repair personnel, including: Based on the content to be repaired, obtain the standard repair process; Acquire images of the restoration process by the restorers; The repair process is determined based on the repair process images and the standard repair process.

5. The intelligent interactive method for ancient building restoration based on image feature recognition as described in claim 1, characterized in that, Step 5: Automatically adjust the angle of the 3D reconstructed ancient building model according to the restoration progress for the restoration personnel to view, including: Get the current viewing angle of the model; Determine the ideal viewing angle for the ancient building model based on the restoration progress; Determine if the current model viewing angle is consistent with the ideal model viewing angle. If they are inconsistent, adjust the current model viewing angle to the ideal model viewing angle for the repair personnel to view.

6. The intelligent interactive method for ancient building restoration based on image feature recognition as described in claim 5, characterized in that, Based on the restoration progress, determine the ideal viewing angle for the ancient building model, including: Determine the current scope of repair based on the repair progress; Determine the current restoration area as a partial model corresponding to the ancient building model; Obtain the angular relationship between the local model and the ancient building model; Determine the ideal viewing angle for the model based on the angular relationships.

7. An intelligent interactive system for the restoration of ancient buildings based on image feature recognition, characterized in that: include: An ancient building image acquisition subsystem is used to acquire the airspace range of the target ancient building; Release the drone within the airspace based on its flight altitude and initial flight angle; Obtain the scan results from the drone; Based on the preset elevation feature extraction template, the elevation features are determined according to the scanning results; Based on the preset elevation difference correlation feature extraction template, the elevation difference correlation features are determined according to the scanning results and elevation features, and the combination relationship between elevation features and elevation difference correlation features is established. Obtain the shooting angle range of the drone and extract the shooting angle range features; Based on the combination relationship, a local elevation model is constructed for the local spatial range of the elevation features corresponding to the combination relationship; Identify the UAV model within the local elevation model; Based on the model position and shooting angle range characteristics of the UAV model, the simulated shooting field of view area in the local elevation model is determined; The basis for extracting elevation difference correlation features corresponding to the combination relationships in the local elevation model is determined by the location model. Identify the reference part model that conforms to the characteristics of the standard reference part model, and use it as the standard reference part model; Obtain the standard reference model corresponding to the flight angle adjustment reference point determination rule; Based on the rules for determining the reference points for flight angle adjustment, determine the reference points for flight angle adjustment in the local elevation model; Determine whether the point for adjusting the flight angle is within the simulated shooting field of view; If not, the corresponding flight angle adjustment reference point will be used as the target point; Connect the target point to the model position of the UAV model by a target line; Determine the boundaries of the two sides of the simulated field of view, and take the boundary of the side of the two sides that is relatively closer to the line connecting the target as the target boundary; Calculate the angle between the target line and the target boundary, and determine the maximum angle corresponding to the local elevation model; The second flight angle is the flight angle based on the maximum angle between the drone's velocity direction and the vertical upward tilt of the horizontal plane. When the drone reaches the local airspace, control the drone to fly at the second flight angle and take pictures; Once all local spatial domains have been photographed, multi-angle images are obtained; The repair content determination subsystem is used to extract image features from multi-angle images and determine the repair content; The repair content annotation subsystem is used for 3D reconstruction based on multi-angle images, to obtain a 3D reconstruction model, to annotate the repair content in the 3D reconstruction model, and to obtain an ancient building model. The repair process acquisition subsystem is used to acquire the repair process of repair personnel; The intelligent interactive subsystem is used to automatically adjust the angle of the 3D reconstructed ancient building model according to the restoration process, so that the restoration personnel can view it.

Images