Three-dimensional modeling method and system for material cultural heritage

Abstract

The embodiment of the present invention relates to the technical field of three-dimensional modeling, and specifically discloses a method and system for three-dimensional modeling of material cultural heritage, wherein a laser radar is used to scan the material cultural heritage to be modeled to obtain point cloud data, an initial three-dimensional model of the material cultural heritage is generated based on the point cloud data, a camera is used to collect color images of the material cultural heritage, the initial three-dimensional model is segmented to obtain multiple sub-blocks, texture information of the multiple sub-blocks is extracted from the color images, and texture mapping is performed on the sub-blocks of the initial three-dimensional model to obtain a target three-dimensional model of the material cultural heritage. The three-dimensional model is used to establish a three-dimensional model of the material cultural heritage, the external dimensions of the material cultural heritage can be intuitively obtained through the three-dimensional model, and the external color of the material cultural heritage can be obtained through the texture information, and the accurate external dimensions and color data of the material cultural heritage can be obtained from the established three-dimensional model to provide an accurate reference for maintenance, protection and reproduction.

Description

Technical Field

[0001] The present invention belongs to the technical field of three-dimensional modeling, and in particular relates to a three-dimensional modeling method and system for material cultural heritage. Background Art

[0002] As objects of cultural relics protection, it is crucial to provide reference for the subsequent restoration and reproduction of material cultural heritage such as ancient buildings and underground ruins.

[0003] In traditional technology, with the development of photography technology, after the discovery or completion of material cultural heritage, the material cultural heritage is usually photographed with a camera to obtain images, and the images are archived as a reference for the restoration and reproduction of the material cultural heritage. However, by taking pictures with a camera, it is impossible to obtain an intuitive visual effect of the material cultural heritage. For example, it is impossible to obtain the specific dimensions of each part of the material cultural heritage. It is difficult to intuitively obtain the external dimensions of the material cultural heritage from the image during repair or reproduction. In addition, in important color areas such as murals and carvings of the material cultural heritage, they can only be identified by referring to the photographed images, and it is difficult to obtain accurate color data. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide a method and system for three-dimensional modeling of material cultural heritage, aiming to solve the problem that it is difficult to intuitively obtain the appearance size and accurate color data of the material cultural heritage by archiving photographed images of the material cultural heritage as a reference.

[0005] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0006] A method for three-dimensional modeling of material cultural heritage, comprising the following steps:

[0007] Use LiDAR to scan the material cultural heritage to be modeled to obtain point cloud data;

[0008] Generating an initial three-dimensional model of the material cultural heritage based on the point cloud data;

[0009] Using a camera to collect color images of the material cultural heritage;

[0010] Segmenting the initial three-dimensional model to obtain a plurality of sub-blocks;

[0011] Extracting texture information of a plurality of sub-blocks from the color image;

[0012] Texture mapping is performed on the sub-blocks of the initial three-dimensional model according to the texture information to obtain the target three-dimensional model of the material cultural heritage.

[0013] As a further solution of the present invention, before using a laser radar to scan the material cultural heritage to be modeled to obtain point cloud data, the following steps are specifically included:

[0014] A reference coordinate system is established with the center of the material cultural heritage to be modeled, and the laser radar and the camera are calibrated with the reference coordinate system to obtain the coordinate conversion relationship between the laser radar and the camera.

[0015] As a further solution of the present invention, a laser radar is used to scan the material cultural heritage to be modeled to obtain point cloud data, which specifically includes the following steps:

[0016] Control the laser radar to scan the material cultural heritage to be modeled to obtain multiple frames of first sub-point cloud data;

[0017] Converting the first sub-point cloud data into second sub-point cloud data in the reference coordinate system through the coordinate transformation relationship;

[0018] The multiple frames of second sub-point cloud data are fused to obtain the point cloud data of the material cultural heritage.

[0019] As a further solution of the present invention, generating an initial three-dimensional model of the material cultural heritage based on the point cloud data specifically includes the following steps:

[0020] Generating an original three-dimensional model of the material cultural heritage based on the point cloud data, and displaying the original three-dimensional model;

[0021] Inputting the original three-dimensional model into a fracture surface detection network to detect a plurality of fracture surface areas in the original three-dimensional model;

[0022] Determine whether the preset conditions are met;

[0023] If so, determining the original three-dimensional model as the initial three-dimensional model of the tangible cultural heritage;

[0024] If not, displaying a selection prompt message for the broken surface area, and responding to the selection operation of the plurality of broken surface areas in the original three-dimensional model to determine the supplementary mining area;

[0025] Scanning the supplementary mining area by the laser radar to obtain supplementary mining point cloud data;

[0026] The original three-dimensional model is repaired using the supplementary point cloud data to obtain a repaired original three-dimensional model, and the original three-dimensional model is input into a fracture detection network to detect multiple areas in the original three-dimensional model.

[0027] As a further solution of the present invention, before the laser radar scans the supplementary mining area to obtain the supplementary mining point cloud data, the following steps are specifically included:

[0028] When a selection operation of a user on a preset area in the original three-dimensional model is received, the preset area is added to the supplementary sampling area.

[0029] As a further solution of the present invention, the initial three-dimensional model is segmented to obtain a plurality of sub-blocks, which specifically includes the following steps:

[0030] Taking the origin of the reference coordinate system as the sphere center, generating the minimum circumscribed sphere of the initial three-dimensional model;

[0031] A spherical mesh of the minimum circumscribed sphere is constructed, and the vertices of each mesh in the spherical mesh are connected with the sphere center to divide the initial three-dimensional model into a plurality of sub-blocks.

[0032] As a further solution of the present invention, extracting texture information of multiple sub-blocks from the color image specifically includes the following steps:

[0033] For each sub-block, determining a texture surface of the sub-block, where the texture surface is an appearance surface of the sub-block predefined in the tangible cultural heritage;

[0034] Converting the plurality of color images captured by the camera into color images in the reference coordinate system according to the conversion relationship between the camera and the reference coordinate system;

[0035] Projecting the multiple color images onto the spherical surface of the minimum circumscribed sphere and removing overlapping areas to obtain at least one layer of panoramic color image of the material cultural heritage;

[0036] An image area covered by the spherical grid in at least one layer of panoramic color image is determined as a texture image of a sub-block obtained by segmenting the spherical grid.

[0037] As a further solution of the present invention, after determining the portion of the panoramic color image included in the spherical grid for segmenting each sub-block as the texture image of the sub-block, the following steps are specifically included:

[0038] Based on the user's merging operation on the sub-blocks, multiple sub-blocks are merged into one sub-block;

[0039] Based on the texture selection operation of the merged sub-block by the user, a target area is selected from the color image as the texture image of the merged sub-block.

[0040] A 3D modeling system for material cultural heritage, specifically including the following modules:

[0041] Point cloud data acquisition module, used to obtain point cloud data by scanning the material cultural heritage to be modeled using laser radar;

[0042] An initial three-dimensional model generation module, used to generate an initial three-dimensional model of the material cultural heritage based on the point cloud data;

[0043] An image acquisition module, used for acquiring color images of the material cultural heritage using a camera;

[0044] A three-dimensional model segmentation module, used for segmenting the initial three-dimensional model into multiple sub-blocks;

[0045] A texture information extraction module, used for extracting texture information of a plurality of sub-blocks from the color image;

[0046] The texture mapping module is used to perform texture mapping on the sub-blocks of the initial three-dimensional model using the texture information to obtain the target three-dimensional model of the material cultural heritage.

[0047] As a further solution of the present invention, the initial three-dimensional model generation module specifically includes the following units:

[0048] An original three-dimensional model generating unit, used to generate an original three-dimensional model of the material cultural heritage based on the point cloud data, and display the original three-dimensional model;

[0049] A fracture surface detection unit, used for inputting the original three-dimensional model into a fracture surface detection network to detect a plurality of fracture surface areas in the original three-dimensional model;

[0050] A condition judgment unit, used to judge whether a preset condition is met, and if so, to execute the initial three-dimensional module determination unit, and if not, to execute the supplementary mining area determination unit;

[0051] An initial three-dimensional model determining unit, used to determine the original three-dimensional model as the initial three-dimensional model of the material cultural heritage;

[0052] a supplementary mining area determination unit, configured to display a selection prompt message of a broken surface area, and determine a supplementary mining area in response to a selection operation on a plurality of broken surface areas in the original three-dimensional model;

[0053] A point cloud data supplementary sampling unit, configured to obtain supplementary point cloud data by scanning the supplementary sampling area through the laser radar;

[0054] The three-dimensional model repairing unit is used to repair the original three-dimensional model using the supplementary point cloud data to obtain the repaired original three-dimensional model and return it to the fracture detection unit.

[0055] Compared with the prior art, the present invention has the following beneficial effects:

[0056] The embodiment of the present invention obtains point cloud data by scanning the material cultural heritage through a laser radar, and collects color images of the material cultural heritage through an audio-visual image. After generating an initial three-dimensional model of the material cultural heritage through the point cloud data, the initial three-dimensional model is segmented to obtain multiple sub-blocks, and texture information of the multiple sub-blocks is extracted from the color image to perform texture mapping on each sub-block to obtain a target three-dimensional model of the material cultural heritage, thereby realizing the establishment of a three-dimensional model of the material cultural heritage. The external dimensions of the material cultural heritage can be intuitively obtained through the three-dimensional model, and the external color of the material cultural heritage can be obtained through the texture information, thereby providing an external accurate terrain dimension and color reference for the maintenance and reproduction of the material cultural heritage. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention.

[0058] Figure 1 A flow chart of a method for three-dimensional modeling of material cultural heritage provided by an embodiment of the present invention is shown.

[0059] Figure 2 A flow chart of generating an initial three-dimensional model through point cloud data in an embodiment of the present invention is shown.

[0060] Figure 3 A flow chart of extracting texture information from a color image in an embodiment of the present invention is shown.

[0061] Figure 4 The application architecture diagram of a 3D modeling system for material cultural heritage provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0063] Figure 1 The flowchart of the 3D modeling method of the material cultural heritage provided by the embodiment of the present invention is shown. Specifically, the 3D modeling method of the material cultural heritage provided by the embodiment of the present invention comprises the following steps:

[0064] S101, using laser radar to scan the material cultural heritage to be modeled to obtain point cloud data.

[0065] In this embodiment, the material cultural heritage can be ancient buildings on the ground or natural scenery defined as material cultural heritage, or various relics under the ground, or various unearthed cultural relics or modern man-made buildings. This embodiment does not limit the material cultural heritage.

[0066] In one embodiment, a three-dimensional model of the material cultural heritage can be reconstructed by using a laser radar and a camera. For example, a laser radar can be set around and inside the material cultural heritage, and a camera can be configured. Then, a reference coordinate system is established with the center of the material cultural heritage to be modeled, and the laser radar and the camera are calibrated with the reference coordinate system to obtain the coordinate conversion relationship between the laser radar and the camera. Specifically, the center of the material cultural heritage can be the geometric center of the material cultural heritage. Taking the material cultural heritage as an ancient building as an example, the center of the material cultural heritage can be the center position of the main body of the ancient building, or a specific position on the central axis of the ancient building. The center of the material cultural heritage and the established reference coordinate system can be determined according to the geometric structure of the material cultural heritage. The embodiment of the present invention does not limit the origin of the reference coordinate system and the direction of the coordinate system.

[0067] After establishing the reference coordinate system, calibration objects can be set on the material cultural heritage, and the laser radar and camera can be calibrated by the calibration objects to obtain the coordinate transformation relationship from the laser radar coordinate system to the reference coordinate system, and the coordinate transformation relationship from the camera coordinate system to the reference coordinate system. The coordinate transformation relationship can refer to the translation and rotation matrix between the two coordinate systems. The specific calibration method can refer to the existing laser radar and camera calibration method, which will not be described in detail here. Before the laser radar and camera collect data, the reference coordinate system is first established to calibrate the laser radar and camera to obtain the coordinate transformation relationship, which can facilitate the subsequent conversion into data in a unified coordinate system when establishing a three-dimensional model and texture mapping.

[0068] In this embodiment, the laser radar scanning path can be set according to the geometric size and layout of the material cultural heritage, and an overlap rate is set between multiple paths to ensure that the laser radar can scan various local positions of the material cultural heritage to fully cover the material cultural heritage. After controlling the laser radar to scan the material cultural heritage to be modeled according to the path, multiple frames of first sub-point cloud data are obtained, and each frame of the first sub-point cloud data obtained by the laser radar scanning is converted into the second sub-point cloud data in the reference coordinate system through the coordinate conversion relationship from the laser radar coordinate system to the reference coordinate system, and then the multiple frames of the second sub-point cloud data are fused to obtain the point cloud data of the material cultural heritage.

[0069] Specifically, each frame of the first sub-point cloud data includes the three-dimensional coordinates of multiple points relative to the coordinate system of the laser radar. The three-dimensional coordinates of each point in the reference coordinate system can be calculated by the translation and rotation matrix from the laser radar coordinate system to the reference coordinate system. Since there is an overlapping area in the scanning range of the laser radar, after obtaining the second sub-point cloud data in the reference coordinate system scanned by the laser radar, the union of multiple frames of the second sub-point cloud data can be taken and deduplication processing can be performed (removing overlapping points) to obtain the point cloud data of the material cultural heritage. Each point in the point cloud data includes the three-dimensional coordinates of the point on the material cultural heritage relative to the reference coordinate system.

[0070] Of course, before coordinate transformation, the first sub-point cloud data may be subjected to denoising, outlier elimination and other processing to improve the quality of the point cloud data of the tangible cultural heritage.

[0071] S102, generating an initial three-dimensional model of the material cultural heritage based on the point cloud data.

[0072] In one embodiment, after obtaining the point cloud data, the point cloud data of the tangible cultural heritage may be input into a 3D model generation software (such as Bentley Pointools) to obtain an initial 3D model of the tangible cultural heritage.

[0073] In another embodiment, generating an initial three-dimensional model of a tangible cultural heritage based on point cloud data may include the following steps:

[0074] S1021, generating an original three-dimensional model of the material cultural heritage based on the point cloud data, and displaying the original three-dimensional model.

[0075] Specifically, the point cloud data can be imported into 3D model generation software such as Bentley Pointools, and the point cloud data can be filtered to remove noise, aligned, segmented and classified in the 3D model software. The processed point cloud data is then converted into a triangular mesh to obtain a 3D model composed of facets, that is, the original 3D model of the material cultural heritage is obtained, and the original 3D model is displayed on a display screen.

[0076] S1022, input the original three-dimensional model into a fracture surface detection network to detect multiple fracture surface areas in the original three-dimensional model.

[0077] The present embodiment can pre-train a broken surface detection network, which can detect areas with broken surfaces in the input three-dimensional model when a three-dimensional model is input. In one embodiment, when training the broken surface detection network, a three-dimensional model sample can be first obtained, in which a plurality of broken surface areas, a first broken surface position of each broken surface area and a first minimum bounding rectangle are marked. The three-dimensional model sample is input into the broken surface detection network to obtain a second broken surface position and a second minimum bounding rectangle of the broken surface area. The loss rate is calculated by the number of broken surface areas, the first broken surface position, the first minimum bounding rectangle, the second broken surface position and the second minimum bounding rectangle. When the training conditions are met, a trained broken surface detection network is obtained. When the training conditions are not met, the network parameters of the broken surface detection network are adjusted by the loss rate, and then the broken surface detection network is continuously trained until the training conditions are met, wherein the training conditions can be that the number of training times reaches a preset number of times, the loss rate is less than a threshold, etc.

[0078] The loss rate is calculated by the following formula:

[0079] ;

[0080] In the above formula, num1 is the number of broken surface areas marked in the three-dimensional model sample, num2 is the number of broken surface areas detected by the broken surface detection model, pos1i is the center position of the first minimum bounding rectangle of the i-th broken surface area marked in the three-dimensional model sample, pos2 is the center position of the second minimum bounding rectangle of the i-th broken surface area detected by the broken surface detection area, Iou(box1,box2) is the intersection and union ratio of the first minimum bounding rectangle box1i and the second minimum bounding rectangle box2i of the i-th broken surface area, a, b, c are weights, and the loss rate is calculated by the number of broken surface areas, the position of the broken surface areas, and the size of the broken surface areas, which can constrain the broken surface detection network to learn the ability to accurately detect the possible broken surfaces, broken surface positions, and broken surface sizes in the three-dimensional model.

[0081] After the fracture detection network is trained, the original three-dimensional model of the generated material cultural heritage can be input into the fracture detection network to obtain the number, position and size of fractures in the original three-dimensional model.

[0082] S1023, determining whether a preset condition is met.

[0083] The preset conditions are used to control the quality of the generated original three-dimensional model. In one embodiment, the preset conditions may be at least one of the number of broken surfaces being less than a preset number, the size of the broken surfaces being less than a preset area, and the distance between the positions of two broken surfaces being greater than a preset distance.

[0084] In another embodiment, the preset condition may also be that the total area of ​​the broken surface is less than a preset area. Of course, computer personnel in this field may also set other preset conditions, and this embodiment does not limit the preset conditions.

[0085] When the preset condition is met, it is determined that the quality of the generated original three-dimensional model meets the requirement, and S1024 can be executed. When the preset condition is not met, it is determined that the quality of the generated original three-dimensional model does not meet the requirement, and S1025 can be executed.

[0086] S1024, determining the original three-dimensional model as the initial three-dimensional model of the tangible cultural heritage.

[0087] When the preset conditions are met, it is determined that the quality of the generated original three-dimensional model has reached the requirements, and the generated original three-dimensional model is the initial three-dimensional model of the material cultural heritage, and texture mapping can be performed on the initial three-dimensional model.

[0088] S1025, displaying a selection prompt message for the broken surface area, and responding to the selection operation on the multiple broken surface areas in the original three-dimensional model to determine the supplementary mining area.

[0089] When the preset conditions are not met, it is determined that the quality of the generated original 3D model does not meet the requirements and the original 3D model needs to be further repaired. Specifically, a selection control for the detected broken surface area can be displayed on the displayed original 3D model. When the user selects the corresponding broken surface area through the selection control, the selected broken surface area is used as a supplementary sampling area, which is used to indicate the re-collection of point cloud data.

[0090] In an optional embodiment, the user can also rotate, translate, zoom, and other operations on the displayed original three-dimensional model to manually evaluate the quality of the original three-dimensional model. When the user finds that the quality of a certain area in the original three-dimensional model does not meet the requirements, the user can select the area, or the user can select a preset area in the original three-dimensional model. The preset area can be an area in the material cultural heritage with high requirements for the three-dimensional model quality. For example, when the material cultural heritage is an ancient building, the preset area can be an area on the ancient building with carvings, screens, ornaments, calligraphy and paintings, etc. that require attention to details. The user can then select the corresponding area to add it to the supplementary sampling area, so as to provide the user with the option of selecting important areas for supplementary sampling to obtain high-quality three-dimensional models of important areas.

[0091] S1026, scanning the supplementary sampling area by laser radar to obtain supplementary sampling point cloud data.

[0092] After determining the supplementary mining area, the position of the laser radar can be changed, or a handheld laser radar can be used to reach the corresponding area of ​​the material cultural heritage and perform laser scanning on the area to obtain the supplementary mining point cloud data.

[0093] S1027, using the supplementary point cloud data to repair the original three-dimensional model to obtain a repaired original three-dimensional model.

[0094] Specifically, after obtaining the supplementary point cloud data, the corresponding broken surface area can be repaired through the point cloud data to achieve local repair of the original three-dimensional model. The specific repair method can be considered as re-entering the three-dimensional model of the broken surface area, which will not be described in detail.

[0095] After the repaired original three-dimensional model is obtained, the process may return to S1022 to perform a crack detection on the repaired original three-dimensional model until a preset condition is met.

[0096] This embodiment performs surface detection and repair on a three-dimensional model established using point cloud data of the tangible cultural heritage to obtain an initial three-dimensional model of the tangible cultural heritage, thereby improving the quality of the three-dimensional model.

[0097] S103, using a camera to collect color images of the tangible cultural heritage.

[0098] Specifically, a camera may be used to capture images of the exterior and visible surfaces of the tangible cultural heritage to obtain multiple color images.

[0099] S104, dividing the initial three-dimensional model into multiple sub-blocks.

[0100] In one embodiment, the minimum circumscribed sphere of the initial three-dimensional model can be generated with the origin of the reference coordinate system as the center of the sphere, a spherical mesh of the minimum circumscribed sphere can be constructed, and the vertices of each mesh in the spherical mesh can be connected to the center of the sphere to divide the initial three-dimensional model into multiple sub-blocks.

[0101] Specifically, equally spaced longitudes and latitudes can be divided on the minimum circumscribed sphere. The intersection of the longitudes and latitudes divides the spherical surface of the minimum circumscribed sphere into multiple square grids. The square cone space formed by the continuity of each direction grid, the four vertices of the square grid and the center of the sphere divides the initial three-dimensional model into multiple sub-blocks, and each sub-block is a quasi-square cone.

[0102] In yet another embodiment, the initial three-dimensional model may also be divided into a plurality of sub-blocks by using cubes of preset sizes. This embodiment does not limit the method of dividing the sub-blocks.

[0103] This embodiment divides the initial three-dimensional model into multiple sub-blocks to determine the texture information of each sub-block. Compared with determining the texture information of the entire three-dimensional model, dividing it into sub-blocks can determine the texture information at a finer granularity, improve the accuracy of the texture information, and obtain more precise color information.

[0104] S105, extracting texture information of multiple sub-blocks from the color image.

[0105] The texture information of the sub-block may be a texture image used for rendering the appearance surface or visible surface of the sub-block in the entire initial three-dimensional model. In one embodiment, extracting the texture information of multiple sub-blocks from the color image may specifically include the following steps:

[0106] S1051. For each sub-block, determine a texture surface of the sub-block, where the texture surface is a predefined appearance surface of the sub-block in the tangible cultural heritage.

[0107] In one embodiment, the texture surface of a sub-block may include the surface of the sub-block away from the center of the minimum circumscribed sphere. Of course, after being divided into sub-blocks, the texture surface of each sub-block can also be determined in response to a user's selection operation. For example, for an ancient building, a divided sub-block may include not only the outermost surface of the ancient building, but also the outer surfaces of internal structures such as screens, murals, window sills, calligraphy and paintings inside the building. The user can select the corresponding surface of the sub-block as the appearance surface.

[0108] S1052: convert the multiple color images captured by the camera into color images in the reference coordinate system according to the conversion relationship between the camera and the reference coordinate system.

[0109] The camera of this embodiment obtains a conversion relationship from the camera coordinate system to the reference coordinate system after calibration before capturing images. The multiple images captured by the camera can be converted into color images in the reference coordinate system through this conversion relationship. For example, the pixel coordinates in the color image can be converted into coordinates in the reference coordinate system through the position of the camera when capturing the image and the conversion relationship from the camera coordinate system to the reference coordinate system, so that the pixels of the converted color image correspond one-to-one to the points on the three-dimensional model.

[0110] S1053, projecting the multiple color images onto the spherical surface of the minimum circumscribed sphere, and removing overlapping areas to obtain at least one layer of panoramic color image of the material cultural heritage.

[0111] There may be overlapping areas in the converted multiple color images. Projecting each color image onto the spherical surface of the minimum circumscribed sphere according to the position of the pixel points in the image in the reference coordinate system is equivalent to projecting each sub-block onto the corresponding spherical grid. After the color image is projected onto the spherical surface, the overlapping area can be removed to obtain at least one layer of panoramic color image of the material cultural heritage. The multi-layer panoramic color images are panoramic color images of the outermost appearance surface to the inner appearance surface of the material cultural heritage from the outside to the center of the sphere. For example, the outermost layer of the ancient building is the outer wall, and from the outer wall to the inside of the ancient building, it can include windows, inner walls, screens, multiple appearance surfaces of indoor objects, etc. Therefore, it can be divided into multiple layers from the outside to the inside according to preset intervals. Accordingly, a multi-layer panoramic color image can be obtained.

[0112] S1054: Determine the image area covered by the spherical grid in at least one layer of the panoramic color image as a texture image of a sub-block obtained by segmenting the spherical grid.

[0113] Since the panoramic color image is projected onto the spherical grid, the spherical grid divides the initial three-dimensional model into sub-blocks, and the area of ​​the panoramic color image covered by each spherical grid is the texture image of the sub-block obtained by dividing the spherical grid.

[0114] In another embodiment, multiple sub-blocks can be merged into one sub-block based on the user's merging operation on the sub-blocks, and the target area can be selected from the color image as the texture image of the merged sub-block based on the user's texture selection operation on the merged sub-block. Exemplarily, when a mural in an ancient building is divided into multiple sub-blocks, the user can merge multiple sub-blocks including the mural, and directly select the color image containing the complete mural as the texture image of the merged sub-block, so as to determine an accurate and complete texture image through the user selection operation, and provide a texture selection channel for special areas such as murals and carvings in ancient buildings, so as to accurately protect the color, content and other information of the special area.

[0115] This embodiment converts the color image and projects it onto a spherical surface, and divides the color image into spherical grids as texture images of each sub-block. The same method as sub-block division is used to obtain accurate texture images of the appearance surface of each sub-block.

[0116] S106, performing texture mapping on the sub-blocks of the initial three-dimensional model according to the texture information to obtain a target three-dimensional model of the tangible cultural heritage.

[0117] After determining the texture image of each sub-block, the texture images of each sub-block in the initial three-dimensional model can be aligned and then subjected to texture mapping processing such as mapping transformation and mapping to obtain a target three-dimensional model of the material cultural heritage. The target three-dimensional model includes both the geometric shape and geometric size information of the material cultural heritage and the color content.

[0118] The embodiment of the present invention obtains point cloud data by scanning the material cultural heritage through a laser radar, and collects color images of the material cultural heritage through an audio-visual image. After generating an initial three-dimensional model of the material cultural heritage through the point cloud data, the initial three-dimensional model is segmented to obtain a plurality of sub-blocks, and texture information of the plurality of sub-blocks is extracted from the color image, so as to perform texture mapping on each sub-block to obtain a target three-dimensional model of the material cultural heritage, thereby realizing the establishment of a three-dimensional model of the material cultural heritage. The external dimensions of the material cultural heritage can be intuitively obtained through the three-dimensional model, and the appearance color of the material cultural heritage can be obtained through the texture information. The accurate appearance dimensions and color data of the material cultural heritage can be obtained from the established three-dimensional model as a reference.

[0119] Figure 4 The application architecture diagram of the tangible cultural heritage three-dimensional modeling system provided by an embodiment of the present invention is shown. The tangible cultural heritage three-dimensional modeling system of this embodiment specifically includes the following units:

[0120] Point cloud data acquisition module 401, used to scan the material cultural heritage to be modeled using laser radar to obtain point cloud data;

[0121] An initial three-dimensional model generating module 402 is used to generate an initial three-dimensional model of the material cultural heritage based on the point cloud data;

[0122] An image acquisition module 403 is used to acquire a color image of the material cultural heritage using a camera;

[0123] A three-dimensional model segmentation module 404 is used to segment the initial three-dimensional model into multiple sub-blocks;

[0124] A texture information extraction module 405, used to extract texture information of multiple sub-blocks from the color image;

[0125] The texture mapping module 406 is used to perform texture mapping on the sub-blocks of the initial three-dimensional model according to the texture information to obtain the target three-dimensional model of the material cultural heritage.

[0126] As an embodiment of the present invention, the following modules are also included:

[0127] The calibration module is used to establish a reference coordinate system with the center of the material cultural heritage to be modeled, and calibrate the laser radar and the camera with the reference coordinate system to obtain the coordinate conversion relationship between the laser radar and the camera.

[0128] As an embodiment of the present invention, the point cloud data acquisition module 401 specifically includes the following units:

[0129] A laser scanning unit, used for controlling the laser radar to scan the material cultural heritage to be modeled to obtain multiple frames of first sub-point cloud data;

[0130] A coordinate conversion unit, used for converting the first sub-point cloud data into second sub-point cloud data in the reference coordinate system through the coordinate conversion relationship;

[0131] The point cloud fusion unit is used to perform fusion processing on multiple frames of second sub-point cloud data to obtain point cloud data of the material cultural heritage.

[0132] As an embodiment of the present invention, the initial 3D model generation module 402 specifically includes the following units:

[0133] An original three-dimensional model generating unit, used to generate an original three-dimensional model of the material cultural heritage based on the point cloud data, and display the original three-dimensional model;

[0134] A fracture surface detection unit, used for inputting the original three-dimensional model into a fracture surface detection network to detect a plurality of fracture surface areas in the original three-dimensional model;

[0135] A condition judgment unit, used to judge whether a preset condition is met, and if so, to execute the initial three-dimensional module determination unit, and if not, to execute the supplementary mining area determination unit;

[0136] An initial three-dimensional model determining unit, used to determine the original three-dimensional model as the initial three-dimensional model of the material cultural heritage;

[0137] a supplementary mining area determination unit, configured to display a selection prompt message of a broken surface area, and determine a supplementary mining area in response to a selection operation on a plurality of broken surface areas in the original three-dimensional model;

[0138] A point cloud data supplementary sampling unit, configured to obtain supplementary point cloud data by scanning the supplementary sampling area through the laser radar;

[0139] The three-dimensional model repairing unit is used to repair the original three-dimensional model using the supplementary point cloud data to obtain the repaired original three-dimensional model and return it to the fracture detection unit.

[0140] As an embodiment of the present invention, the initial 3D model generation module 402 specifically further includes the following units:

[0141] The selection unit is configured to add the preset area to the supplementary sampling area when receiving a selection operation of the user on the preset area in the original three-dimensional model.

[0142] As an embodiment of the present invention, the 3D model segmentation module 404 specifically includes the following units:

[0143] A sphere generating unit, used to generate a minimum circumscribed sphere of the initial three-dimensional model with the origin of the reference coordinate system as the sphere center;

[0144] The segmentation unit is used to construct a spherical mesh of the minimum circumscribed sphere and connect the vertices of each mesh in the spherical mesh with the center of the sphere to segment the initial three-dimensional model into a plurality of sub-blocks.

[0145] As an embodiment of the present invention, the texture information extraction module 405 specifically includes the following units:

[0146] A texture surface determination unit, configured to determine, for each sub-block, a texture surface of the sub-block, wherein the texture surface is an appearance surface of the sub-block predefined in the tangible cultural heritage;

[0147] An image conversion unit, configured to convert the plurality of color images acquired by the camera into color images in the reference coordinate system according to a conversion relationship between the camera and the reference coordinate system;

[0148] A projection unit, used for projecting the plurality of color images onto the spherical surface of the minimum circumscribed sphere and removing overlapping areas to obtain at least one layer of panoramic color image of the material cultural heritage;

[0149] The texture image determination unit is used to determine the image area covered by the spherical grid in at least one layer of panoramic color image as the texture image of the sub-block obtained by segmenting the spherical grid.

[0150] As an embodiment of the present invention, the texture information extraction module 405 specifically further includes the following units:

[0151] A sub-block merging unit, configured to merge multiple sub-blocks into one sub-block based on a merging operation of the sub-blocks by a user;

[0152] The texture selection unit is used to select a target area from the color image as a texture image of the merged sub-block based on a texture selection operation of the merged sub-block by a user.

[0153] It should be understood that, although each step in the flow chart of each embodiment of the present invention is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0154] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0155] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0156] The above embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention patent shall be subject to the attached claims.

[0157] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A three-dimensional modeling method for material cultural heritage, It is characterized in that The specific steps include: Establishing a reference coordinate system with the center of the material cultural heritage to be modeled, and calibrating the laser radar and the camera with the reference coordinate system to obtain the coordinate transformation relationship between the laser radar and the camera; Use LiDAR to scan the material cultural heritage to be modeled to obtain point cloud data; Generating an initial three-dimensional model of the material cultural heritage based on the point cloud data; Using a camera to collect color images of the material cultural heritage; Segmenting the initial three-dimensional model to obtain a plurality of sub-blocks; Extracting texture information of a plurality of sub-blocks from the color image; Performing texture mapping on the sub-blocks of the initial three-dimensional model according to the texture information to obtain a target three-dimensional model of the material cultural heritage; The initial three-dimensional model is segmented to obtain a plurality of sub-blocks, specifically comprising the following steps: Taking the origin of the reference coordinate system as the sphere center, generating the minimum circumscribed sphere of the initial three-dimensional model; Constructing a spherical mesh of the minimum circumscribed sphere, and connecting the vertex of each mesh in the spherical mesh with the sphere center to divide the initial three-dimensional model into a plurality of sub-blocks; Extracting texture information of multiple sub-blocks from the color image specifically includes the following steps: For each sub-block, determining a texture surface of the sub-block, wherein the texture surface is an appearance surface of the sub-block predefined in the tangible cultural heritage, and the appearance surface includes an outermost surface of the sub-block and an outer surface of an internal structure of the sub-block; Converting the plurality of color images captured by the camera into color images in the reference coordinate system according to the conversion relationship between the camera and the reference coordinate system; Projecting the multiple color images onto the spherical surface of the minimum circumscribed sphere and removing overlapping areas to obtain at least one layer of panoramic color image of the material cultural heritage, wherein the material cultural heritage is divided into multiple layers from the outside to the inside according to preset intervals, and the multiple layers of panoramic color images are panoramic color images of the outermost appearance surface to the inner appearance surface of the material cultural heritage from the outside to the center of the sphere; An image area covered by the spherical grid in at least one layer of panoramic color image is determined as a texture image of a sub-block obtained by segmenting the spherical grid.

2. The three-dimensional modeling method of material cultural heritage according to claim 1, It is characterized in that The point cloud data is obtained by scanning the material cultural heritage to be modeled using LiDAR, which specifically includes the following steps: Control the laser radar to scan the material cultural heritage to be modeled to obtain multiple frames of first sub-point cloud data; Converting the first sub-point cloud data into second sub-point cloud data in the reference coordinate system through the coordinate transformation relationship; The multiple frames of second sub-point cloud data are fused to obtain the point cloud data of the material cultural heritage.

3. The three-dimensional modeling method of material cultural heritage according to claim 1, It is characterized in that Generating an initial three-dimensional model of the material cultural heritage based on the point cloud data specifically includes the following steps: Generating an original three-dimensional model of the material cultural heritage based on the point cloud data, and displaying the original three-dimensional model; Inputting the original three-dimensional model into a fracture surface detection network to detect a plurality of fracture surface areas in the original three-dimensional model; Determine whether the preset conditions are met; If so, determining the original three-dimensional model as the initial three-dimensional model of the tangible cultural heritage; If not, displaying a selection prompt message for the broken surface area, and responding to the selection operation of the plurality of broken surface areas in the original three-dimensional model to determine the supplementary mining area; Scanning the supplementary mining area by the laser radar to obtain supplementary mining point cloud data; The original three-dimensional model is repaired using the supplementary point cloud data to obtain a repaired original three-dimensional model, and the original three-dimensional model is input into a fracture detection network to detect multiple areas in the original three-dimensional model.

4. The three-dimensional modeling method of material cultural heritage according to claim 3, It is characterized in that Before the laser radar is used to scan the supplementary mining area to obtain the supplementary mining point cloud data, the following steps are specifically included: When a selection operation of a user on a preset area in the original three-dimensional model is received, the preset area is added to the supplementary sampling area.

5. The three-dimensional modeling method of material cultural heritage according to claim 1, It is characterized in that After determining the portion of the panoramic color image included in the spherical grid for segmenting each sub-block as the texture image of the sub-block, the method further includes the following steps: Based on the user's merging operation on the sub-blocks, multiple sub-blocks are merged into one sub-block; Based on the texture selection operation of the merged sub-block by the user, a target area is selected from the color image as the texture image of the merged sub-block.

6. A 3D modeling system for material cultural heritage, It is characterized in that Specifically includes the following modules: A calibration module is used to establish a reference coordinate system with the center of the material cultural heritage to be modeled, and calibrate the laser radar and the camera with the reference coordinate system to obtain the coordinate conversion relationship between the laser radar and the camera; Point cloud data acquisition module, used to obtain point cloud data by scanning the material cultural heritage to be modeled using laser radar; An initial three-dimensional model generation module, used to generate an initial three-dimensional model of the material cultural heritage based on the point cloud data; An image acquisition module, used for acquiring color images of the material cultural heritage using a camera; A three-dimensional model segmentation module, used for segmenting the initial three-dimensional model into multiple sub-blocks; A texture information extraction module, used for extracting texture information of a plurality of sub-blocks from the color image; A texture mapping module, used to perform texture mapping on the sub-blocks of the initial three-dimensional model according to the texture information to obtain a target three-dimensional model of the material cultural heritage; The 3D model segmentation module specifically includes the following units: A sphere generating unit, used to generate a minimum circumscribed sphere of the initial three-dimensional model with the origin of the reference coordinate system as the sphere center; A segmentation unit, used for constructing a spherical mesh of the minimum circumscribed sphere, and connecting the vertex of each mesh in the spherical mesh with the sphere center to segment the initial three-dimensional model into a plurality of sub-blocks; The texture information extraction module specifically includes the following units: A texture surface determination unit, configured to determine, for each sub-block, a texture surface of the sub-block, wherein the texture surface is an appearance surface of the sub-block predefined in the tangible cultural heritage, and the appearance surface includes an outermost surface of the sub-block and an outer surface of an internal structure of the sub-block; An image conversion unit, configured to convert the plurality of color images acquired by the camera into color images in the reference coordinate system according to a conversion relationship between the camera and the reference coordinate system; A projection unit, used for projecting the plurality of color images onto the spherical surface of the minimum circumscribed sphere and removing overlapping areas to obtain at least one layer of panoramic color image of the material cultural heritage, wherein the material cultural heritage is divided into a plurality of layers from the outside to the inside according to preset intervals, and the multi-layer panoramic color image is a panoramic color image of the outermost appearance surface to the inner appearance surface of the material cultural heritage from the outside to the center of the sphere; The texture image determination unit is used to determine the image area covered by the spherical grid in at least one layer of panoramic color image as the texture image of the sub-block obtained by segmenting the spherical grid.

7. The three-dimensional modeling system of material cultural heritage according to claim 6, It is characterized in that The initial 3D model generation module specifically includes the following units: An original three-dimensional model generating unit, used to generate an original three-dimensional model of the material cultural heritage based on the point cloud data, and display the original three-dimensional model; A fracture surface detection unit, used for inputting the original three-dimensional model into a fracture surface detection network to detect a plurality of fracture surface areas in the original three-dimensional model; A condition judgment unit, used to judge whether a preset condition is met, and if so, to execute the initial three-dimensional module determination unit, and if not, to execute the supplementary mining area determination unit; An initial three-dimensional model determining unit, used to determine the original three-dimensional model as the initial three-dimensional model of the material cultural heritage; a supplementary mining area determination unit, configured to display a selection prompt message of a broken surface area, and determine a supplementary mining area in response to a selection operation on a plurality of broken surface areas in the original three-dimensional model; A point cloud data supplementary sampling unit, configured to obtain supplementary point cloud data by scanning the supplementary sampling area through the laser radar; The three-dimensional model repairing unit is used to repair the original three-dimensional model using the supplementary point cloud data to obtain the repaired original three-dimensional model and return it to the fracture detection unit.

Images