Cultural relic activation method based on improved feature point description method and related device

Through the improved feature point description method, combined with color and texture characteristics, the problem of insufficient description of cultural relics surface features is solved, the feature matching degree and cultural relics activation effect are improved, and the user has a richer experience.

CN119068216APending Publication Date: 2024-12-03HUNAN UNIV
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202411273241.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-11
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The existing feature point descriptors cannot effectively describe the rich color and texture characteristics of the cultural relics surface, resulting in low matching of features and affecting the activation effect of cultural relics.

Method used

An improved feature point description method is adopted, combined with traditional BRIEF descriptors, color features and texture features, and improved feature descriptors are constructed for multi-scale feature point detection and matching, and dynamic texture mapping is realized to activate cultural relics.

Benefits of technology

It improves the matching of features, enhances the activation effect of cultural relics, and allows users to experience and understand the historical and cultural values ​​of cultural relics more deeply.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068216A_ABST
    Figure CN119068216A_ABST
Patent Text Reader

Abstract

The invention discloses a cultural relic activation method based on an improved feature point description method and a related device, and relates to the technical field of computer vision, and the method comprises the steps: obtaining an observation image of a target cultural relic digital twin model at a current observation angle; detecting feature points under different scales in the current observation image by using a multi-scale feature point detection method; performing feature description on the feature points in the current observation image by using an improved feature point description method to obtain an improved feature descriptor; performing feature matching and screening according to the improved feature descriptors corresponding to the feature points in the current observation image and the previous observation image to obtain a feature matching result; and performing dynamic texture mapping in the target cultural relic digital twin model according to a feature matching result to realize cultural relic activation. The improved feature point description method is utilized to solve the problem of low feature matching degree caused by the fact that a common descriptor cannot describe more information on the surface of a cultural relic.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a method for activating cultural relics based on an improved feature point description method and a related device. Background Art

[0002] Cultural relic revitalization is to digitize traditional cultural relics through modern technology, and use technologies such as virtual reality, augmented reality and interactive design to make cultural relics more interactive, visual and experiential, so that people can understand historical culture more intuitively and have a stronger interest and understanding of it. This technology not only helps to better preserve and pass on cultural relics, but also allows the public to have a deeper understanding and experience of the historical, cultural and artistic value carried by cultural relics.

[0003] Cultural relic revitalization usually involves the following work: Use 3D scanning, photogrammetry and other technologies to digitally model cultural relics, generate high-precision three-dimensional models, and accurately present the appearance, structure and details of cultural relics. Combined with game design, interactive design and other technologies, rich interactive functions are designed for cultural relic revitalization projects, such as click to expand, drag to rotate, zoom in and out, etc., so that users can freely explore all aspects of cultural relics and enhance their sense of participation and experience.

[0004] Among them, feature point matching is an important technology in computer vision and image processing, which is used to find similar points or areas between different images. This process is crucial for various applications, such as image stitching, object recognition, 3D reconstruction, image tracking, etc. It includes steps such as feature point detection, feature description, feature matching and filtering. Computer vision has made many advances in cultural relic activation and other fields, but the surface of cultural relics has not only shape features, but also color and texture features. Ordinary descriptors cannot describe all the information, which may lead to low feature matching and affect the effect of cultural relic activation. Summary of the invention

[0005] The purpose of this application is to provide a method and related device for revitalizing cultural relics based on an improved feature point description method, which uses the improved feature point description method to solve the problem that ordinary descriptors cannot describe more information on the surface of cultural relics, resulting in low feature matching.

[0006] To achieve the above objectives, this application provides the following solutions:

[0007] In a first aspect, the present application provides a method for revitalizing cultural relics based on an improved feature point description method, comprising:

[0008] Obtain the observation image of the digital twin model of the target cultural relic at the current observation angle;

[0009] For the currently observed image, using a multi-scale feature point detection method to detect feature points at different scales in the currently observed image;

[0010] The improved feature point description method is used to describe the feature points at each scale in the current observed image, and an improved feature descriptor corresponding to the feature points at each scale is obtained; the improved feature point description method is based on the traditional BRIEF descriptor and combines color features and texture features to form an improved feature descriptor;

[0011] Perform feature matching and screening according to the improved feature descriptors corresponding to the feature points at each scale in the current observation image and the previous observation image, to obtain a feature matching result;

[0012] According to the feature matching results, dynamic texture mapping is performed in the digital twin model of the target cultural relic to realize cultural relic activation.

[0013] In a second aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-mentioned method for revitalizing cultural relics based on the improved feature point description method.

[0014] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned method for revitalizing cultural relics based on the improved feature point description method.

[0015] In a fourth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above-mentioned cultural relics activation method based on the improved feature point description method.

[0016] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0017] The present application provides a method for revitalizing cultural relics based on an improved feature point description method and related devices, introduces an improved feature point description method, and combines color features and texture features on the basis of the traditional BRIEF descriptor to form an improved feature descriptor. The improved feature descriptor includes not only shape features but also color and texture features, and can describe more information on the surface of cultural relics, making the feature matching degree higher, and further improving the cultural relic activation effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 This is an application environment diagram of a method for revitalizing cultural relics based on an improved feature point description method in an embodiment of the present application;

[0020] Figure 2 A flowchart of a method for revitalizing cultural relics based on an improved feature point description method provided in one embodiment of the present application;

[0021] Figure 3 Feature point detection and matching results for different scenarios provided in one embodiment of the present application;

[0022] Figure 4 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0024] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0025] The cultural relics activation method based on the improved feature point description method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set up separately, integrated on the server 104, or placed on the cloud or other servers. The terminal 102 can send the observed image of the target cultural relic digital twin model at the current observation angle to the server 104. After the server 104 receives the observed image of the target cultural relic digital twin model to be processed at the current observation angle, the server 104 uses a multi-scale feature point detection method to detect the feature points at different scales in the current observation image; the feature points at each scale in the current observation image are described by the improved feature point description method, and the improved feature descriptor corresponding to the feature points at each scale is obtained; according to the improved feature descriptor corresponding to the feature points at each scale in the current observation image and the previous frame observation image, feature matching and screening are performed to obtain feature matching results; according to the feature matching results, dynamic texture mapping is performed in the target cultural relic digital twin model to realize cultural relic activation. The server 104 can feedback the cultural relic activation effect to the terminal 102. In addition, in some embodiments, the cultural relic activation method based on the improved feature point description method can also be implemented independently by the server 104 or the terminal 102. For example, the terminal 102 can directly perform feature point detection, feature description, feature matching and screening, and dynamic texture mapping on the observation image of the target cultural relic digital twin model at the current observation angle, or the server 104 can obtain the observation image of the target cultural relic digital twin model at the current observation angle from the data storage system, and perform feature point detection, feature description, feature matching and screening, and dynamic texture mapping.

[0026] The terminal 102 may be, but is not limited to, various desktop computers, laptop computers, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart vehicle-mounted devices, etc. The portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers, or may be a cloud server.

[0027] In an exemplary embodiment, Figure 2 As shown, a method for revitalizing cultural relics based on an improved feature point description method is provided. The method is executed by a computer device, and can be executed by a computer device such as a terminal or a server alone, or by a terminal and a server together. In the embodiment of the present application, the method is applied to Figure 1 The server 104 in the example is used as an example to illustrate, including the following steps 201 to 205. Among them:

[0028] Step 201: Obtain an observation image of the digital twin model of the target cultural relic at the current observation angle.

[0029] In the present invention, for the construction of the digital twin model of the target cultural relic, multiple photos of the target cultural relic can be taken at different angles and under different lighting conditions to better model the digital twin of the cultural relic.

[0030] The observed image obtained in this step is not used to show the user the cultural relic activation effect, but is used for feature point detection and feature description by computer equipment. It will be shown to the user only after dynamic texture mapping in the following step.

[0031] Step 202: for the currently observed image, a multi-scale feature point detection method is used to detect feature points at different scales in the currently observed image.

[0032] Step 203, using an improved feature point description method to describe the feature points at each scale in the current observed image, and obtain an improved feature descriptor corresponding to the feature points at each scale; the improved feature point description method is based on the traditional BRIEF descriptor and combines color features and texture features to form an improved feature descriptor.

[0033] Step 204 , performing feature matching and screening according to the improved feature descriptors corresponding to the feature points at each scale in the current observation image and the previous observation image, to obtain a feature matching result.

[0034] The process of feature point detection and feature description will be performed for each observation image under each observation angle. For two adjacent observation images, when feature matching is performed, there is no need to perform feature point detection and feature description on the observation image of the previous frame. The improved feature description sub-data corresponding to the previous feature points can be directly retrieved for feature matching.

[0035] Step 205 , performing dynamic texture mapping in the digital twin model of the target cultural relic according to the feature matching result to realize cultural relic activation.

[0036] Implement the above-mentioned steps 201 to 205, use the improved BRIEF descriptor to describe the feature points, combine the color histogram and local binary pattern (LBP) around the feature points with the BRIEF descriptor to jointly describe the feature points, therefore, it is possible to solve the problem that ordinary descriptors cannot fully describe the features of cultural relics, resulting in a low degree of matching. In addition, the present application can also realize multi-scale feature point detection by constructing an image pyramid, thus solving the problem that the current conventional feature point detection method cannot fully capture many detail points. With the support of computer vision technology, the method of the present invention can utilize advanced digital twin technology to realize the digital reproduction and activation of cultural relics. By establishing a digital twin model of cultural relics and activating the patterns on the surface of cultural relics, a high degree of restoration and in-depth interpretation of cultural relics can be achieved. This method has important application prospects in the protection, display and research of cultural relics, and provides new possibilities for the inheritance and development of cultural relics.

[0037] In another exemplary embodiment of the present application, in step 203, the feature point recognition operation in this step is implemented by a multi-scale feature point detection method, which can be further explained as being obtained based on an improved multi-scale Harris corner feature point detection method. The surface of cultural relics often has rich textures and details, such as carvings, paintings, weathering marks, etc. Conventional feature point detection algorithms may not be able to fully capture these details. Therefore, the present invention adopts a multi-scale feature point detection method. First, the same image is scaled to construct an image pyramid, and the image of each layer of the image pyramid is represented by the following formula:

[0038]

[0039] Among them, Q is the input image, that is, the observed image, Q i is the scaled image of the i-th level of the image pyramid, s is the scaling factor, Indicates that the image Q is proportional Scaling. After testing, the best feature point detection effect is achieved when the scaling factor is 1.5 and the number of image pyramid layers is 4.

[0040] Secondly, the corner points of each scaled image in the image pyramid are calculated to obtain the feature points in each scaled image.

[0041] Therefore, based on the above feature point recognition operation process, it is concluded that for the current observed image, a multi-scale feature point detection method is used to detect feature points at different scales in the current observed image, specifically including:

[0042] (1) The currently observed image is scaled according to different scaling ratios to obtain an image pyramid corresponding to the currently observed image; each layer of the image pyramid corresponds to a scaled image of a scaling ratio; each layer of the image pyramid corresponds to a scale; and one observed image corresponds to multiple scaled images of different scales.

[0043] (2) Detect feature points in each zoomed image corresponding to the current observed image.

[0044] In another exemplary embodiment of the present application, in the above step (2), for a given image and a neighborhood window of a fixed size, the sum of the squares of the pixel differences before and after the window is translated is calculated, corresponding to the square and sum part in the following formula, where W in the formula refers to the sliding window.

[0045] E(u,v)=Σ (x,y)∈W w(x,y)[I(x+u,y+v)-I(x,y)] 2 ;

[0046] Among them, (x, y) is the position coordinate before translation, (x+u, y+v) is the position coordinate after translation, (u, v is the offset, and I(x, y) represents the pixel value at the pixel point (x, y) in the image. The window weighting function w(x, y) can be a mean function or a Gaussian function. According to Taylor expansion, the first-order approximation of the image after window translation can be obtained, so E(u, v) can be transformed into:

[0047]

[0048] Among them, E(u,v) refers to the grayscale change of the pixel in the window before and after sliding. M(x,y) represents the autocorrelation matrix at the pixel point (x,y), I x Represents the pixel gradient of the pixel in the x direction, I y Represents the pixel gradient of the pixel in the y direction.

[0049] When determining the corner point, there is no need to specifically calculate the eigenvalue of the matrix M, but the following formula is used to approximate the corner point response value.

[0050] R=detM-α(traceM) 2 ;

[0051] detM=λ1λ2=AB-C 2 ;

[0052] traceM=λ1+λ2=A+B;

[0053] Where detM is the determinant of the matrix M, traceM is the trace of the matrix M, and α is a constant, usually ranging from 0.04 to 0.06.

[0054] Calculate the gradient of the pixel point (x, y) of the image in the x direction and y direction as:

[0055]

[0056] Use the window Gaussian function to I x I y Gaussian weighting is performed to generate the matrix M, and the Harris response value R of each pixel in the image before translation is calculated. A threshold T is set, and R that is less than the threshold T is set to zero. Non-maximum suppression is performed in a neighborhood of a fixed window size (5×5), and the local maximum point is the corner point in the image, and the detected corner point is the feature point.

[0057] Therefore, step (2) of detecting feature points in each zoomed image corresponding to the current observed image specifically includes:

[0058] (21) For each scaled image, the pixel gradient of each pixel point in the preset sliding window in the scaled image in the x direction and the y direction is calculated.

[0059] (22) Calculate the autocorrelation matrix of each pixel point according to the pixel gradient of each pixel point in the x direction and the y direction in the preset sliding window.

[0060] (23) Calculate the Harris response value of each pixel point in the preset sliding window according to the autocorrelation matrix.

[0061] (24) From the pixels whose Harris response values ​​are greater than a preset threshold, the pixels whose Harris response values ​​are local maximum values ​​are selected as feature points in the scaled image.

[0062] In another exemplary embodiment of the present application, in step 204, this step uses an improved BRIEF descriptor, which is combined with the color histogram around the feature point and the local binary pattern (LBP) feature to form a new descriptor: first calculate the color histogram of the area around each feature point, and for each feature point, extract an image block P of size×size around it e . Assume that the coordinates of the feature point are (x e ,y e ), image block P e It can be expressed as:

[0063]

[0064] For each image block Pe , calculate the color histogram of the blue channel, green channel and red channel respectively:

[0065]

[0066] Where I(m,n,c) is the pixel value of the image at position (m,n) and channel c (blue, green or red). M and N represent the width and height of the image block; w represents the bin width of the color histogram; and k represents a bin index in the color histogram. Specifically, It is to classify the pixel value I(m,n,c) into a bucket, w is the bucket width, Indicates a round-down operation. For example, if the pixel value is 120 and the bucket width w is 50, then This means that the pixel value 120 is placed in the second bucket. is the Kronecker delta function. It It returns 1 if the pixel value belongs to the current bucket k, otherwise it returns 0. This is used to check whether the pixel value belongs to the current bucket k. The final h k is the number of pixels in the kth bucket, that is, the frequency of the kth bucket in the histogram. The frequency of each bucket can be obtained by accumulating the counts of all pixels in the image block.

[0067] Secondly, calculate the LBP features of the area around each feature point. The LBP feature determines the value of 1 or 0 based on the size of the adjacent pixel points compared with the central pixel point (referring to the feature point being processed). That is, if the value of the adjacent pixel is greater than or equal to the value of the central pixel, the value is 1; if it is less than the value, the value is 1. Given a grayscale image, first calculate each pixel position (x e ,y e ). Assuming that radius r and P neighborhood points are used, the calculation formula of LBP value is as follows:

[0068]

[0069] (x p ,y p ) is the center pixel (x e ,y e ) is the coordinates of P neighboring points with a radius of r around . s(*) is a sign function, when *≥0, s(*)=1, otherwise s(*)=0.

[0070] Next, the BRIEF descriptor is calculated for each feature point. The BRIEF descriptor is constructed by comparing the pixel pairs around the feature point. First, a fixed window or neighborhood (e.g., a 31×31 pixel window) is selected around a feature point. Some pixel pairs (s, t) are randomly selected within this window. Each pixel pair consists of a relative position (xs ,y s ) and (x t ,y t )composition.

[0071] For each pixel pair (s, t), compare their grayscale values. If the brightness of pixel s is greater than the grayscale of pixel t, the binary bit of the pair is 1, otherwise it is 0.

[0072]

[0073] For each feature point, concatenate the bit strings of all pixel pairs to form a descriptor vector D e . Descriptor D e It can be represented as a binary bit string of length n:

[0074] D e =[b1,b2,…,b f ,…,b F ];

[0075] Among them, each b f is a binary value (0 or 1) indicating the comparison result of the fth pixel pair. F represents the number of pixel pairs (s, t).

[0076] Finally, the three descriptive indicators obtained above are combined.

[0077] Therefore, based on the above feature description process, step 204 uses an improved feature point description method to perform feature description on the feature points at each scale in the current observed image, and obtains an improved feature descriptor corresponding to the feature points at each scale, specifically including:

[0078] (a1) for each feature point at each scale in the currently observed image, a color histogram in a first preset area around the feature point is calculated; the first preset area includes the corresponding feature point; and the color histogram is used to reflect the color characteristics of the feature point.

[0079] (a2) for each feature point at each scale in the currently observed image, calculating the LBP value in a second preset area around the feature point; the second preset area includes the corresponding feature point; and the LBP value is used to reflect the texture feature of the feature point.

[0080] (a3) For each feature point at each scale in the currently observed image, a BRIEF feature descriptor in a third preset area around the feature point is calculated; the third preset area includes the corresponding feature point.

[0081] (a4) For each feature point at each scale in the currently observed image, the color histogram, LBP value and BRIEF feature descriptor of the feature point are combined to obtain an improved feature descriptor corresponding to the feature point.

[0082] In order to adapt to the subsequent feature matching algorithm, it is necessary to convert various description indicators into float32 type. In this way, for each layer of the image pyramid, the descriptors of each feature point are connected in rows to form a large descriptor array. Assume that the shape of the descriptor array on each layer of the image pyramid is (N i ,138), where N i is the number of feature points detected on the i-th layer. For each layer of the image pyramid, the shape of the stacked descriptor array will be:

[0083]

[0084] Among them, L represents the number of layers of the image pyramid, and the number of columns of the array 138 comes from the total length of the descriptor, including the BRIEF descriptor length: 32, the color histogram length: 32×3, and the LBP descriptor length: 10.

[0085] For some feature points, there is no corresponding descriptor. In order to calculate the subsequent feature matching algorithm, for feature points that do not have descriptor information, an empty descriptor array is initialized. Its shape is 0,138):

[0086]

[0087] In another exemplary embodiment of the present application, in step 205, the feature points obtained above and the corresponding descriptors are matched and screened, the KNN matching method is used for feature point matching, and the ratio detection and RANSAC methods are used to screen the matched feature points to ensure the geometric consistency of the matching. Before feature matching, the detected feature points need to be adjusted to the coordinate system of the original image according to the scaling factor of the corresponding layer: (x, y) original =(x·s i ,y·s i ), where (x,y) original is the adjusted coordinate, (x, y) is the coordinate of the feature point detected in the i-th layer image, multiplied by the scaling factor s according to the formula i Then it can correspond to the image before scaling.

[0088] In the KNN matching method, the distance between objects is calculated as the non-similarity index between objects to avoid the matching problem between objects. The improved feature descriptor of each feature point is a feature vector, which is usually high-dimensional. Assume that the feature point descriptor set in image A is The feature point descriptor set in image B is First, calculate the improved descriptor of each feature point in image A The improved descriptor for each feature point in image B Distance between: Then sort them and select the K feature points with the smallest distance, and obtain the K feature points in image B that are closest to a feature point in image A. Next, perform ratio detection on the feature points obtained by the KNN matching method, traverse all matching pairs, and apply ratio test to screen good matches. Finally, use the RANSAC method to screen the matching point pairs obtained after ratio detection and screening again. Therefore, step 205 performs feature matching and screening according to the improved feature descriptors corresponding to the feature points at each scale in the current observation image and the previous observation image, and obtains the feature matching results, which specifically include:

[0089] (b1) Restoring the feature points at each scale corresponding to the current observed image to the coordinate system of the original image to obtain a number of restored feature points; the original image refers to the observed image before scaling.

[0090] (b2) For two adjacent frames of observation images, a preliminary matching point for each restored feature point in the current observation image is determined using a KNN matching method; the preliminary matching point refers to the K restored feature points in the previous frame of observation image that are closest to the restored feature point in the current observation image.

[0091] (b3) Each restored feature point in the current observed image is combined with the corresponding K preliminary matching points to form K preliminary matching point pairs; the K preliminary matching point pairs include a target preliminary matching point pair and a non-target preliminary matching point pair; the target preliminary matching point pair refers to the preliminary matching point pair with the smallest distance between the restored feature point and the preliminary matching point; the matching point pairs in the preliminary matching point pairs excluding the target preliminary matching point pair are the non-target preliminary matching point pairs.

[0092] (b4) According to the distance between the matching points in the target preliminary matching point pair and the distance between the matching points in each of the non-target preliminary matching point pairs, a ratio detection method is applied to screen the target preliminary matching point pairs corresponding to the restored feature points in the current observation image, and obtain the preliminary screened matching point pairs corresponding to each restored feature point in the current observation image; one restored feature point in the current observation image corresponds to one preliminary screened matching point pair.

[0093] For each matching pair (X,Y j ), if the ratio test condition is met, it is added to the good_matches list, which is specifically expressed as:

[0094] good matches = {X | X, Y j ∈matches,d(X) <ratio×d(Y1),d(X)<ratio×

[0095] d(Y2),...,d(X) <ratio×d(Y j )};

[0096] Among them, good matches represents the set list of preliminary screened matching point pairs; matches represents the set list of K preliminary matching point pairs; X represents the target preliminary matching point pair; Y j represents the jth non-target preliminary matching point pair; ratio represents the scaling factor; d(X) represents the distance between the matching points in the target preliminary matching point pair; d(Y j ) represents the distance between the matching points in the jth non-target preliminary matching point pair.

[0097] (b5) using the RANSAC method to screen the preliminary screened matching point pairs corresponding to all restored feature points in the current observed image, and obtain the matching point pairs after secondary screening; the matching point pairs after secondary screening are the feature matching results.

[0098] In another exemplary embodiment of the present application, the RANSAC method is an iterative method for estimating mathematical model parameters by using observed data points, which can effectively filter out incorrect matching point pairs and retain only correct matching point pairs. First, multiple matching point pairs (for example, 4 matching point pairs) are randomly selected from the matching pair set. The homography matrix H is estimated by the 4 matching point pairs, so that: x′ g ≈Hx g ; For all matching pairs Calculate the transformation error: d(x g ,x′ g )=||x g ′-Hx g ||; G is the number of matching point pairs obtained by the ratio detection method.

[0099] Points with statistical errors less than the threshold ε are regarded as inliers. Then, multiple pairs of matching points are randomly selected from the inliers, and the above steps are repeated until the preset number of iterations is reached to find the optimal homography matrix H. Based on the final homography matrix H, a mask is generated.

[0100]

[0101] Where mask(g) indicates whether the g-th pair of matching points is an inlier. The inliers obtained by the optimal homography matrix H are the final matching point pairs selected by the RANSAC method.

[0102] Therefore, step (b5), using the RANSAC method to screen all the restored feature points in the current observed image, the initial screening matching point pairs corresponding to the obtained matching point pairs after the secondary screening, specifically includes:

[0103] (c1) randomly extracting a plurality of matching point pairs from all the preliminary screened matching point pairs corresponding to all the restored feature points in the current observed image.

[0104] (c2) estimating a homography matrix using the extracted matching point pairs; the homography matrix is ​​used to represent the relationship between the matching points in the extracted matching point pairs.

[0105] (c3) Calculate the transformation error for each preliminary screening matching point pair using the currently estimated homography matrix, and obtain the transformation error corresponding to each preliminary screening matching point pair.

[0106] (c4) randomly extracting a plurality of matching point pairs from the preliminary screened matching point pairs whose transformation errors are less than a preset transformation error threshold, and returning to step "estimating the homography matrix using the extracted matching point pairs".

[0107] (c5) When the number of iterations reaches a preset number of iterations and an optimal homography matrix is ​​obtained, or when the homography matrix calculated in the next iteration process is smaller than the homography matrix of the previous iteration process, the homography matrix of the previous iteration process is regarded as the optimal homography matrix.

[0108] (c6) Use the optimal homography matrix to calculate the transformation error corresponding to each pair of preliminary screened matching points.

[0109] (c7) Screening out preliminary screening matching point pairs whose target transformation error is less than a preset transformation error threshold, and obtaining matching point pairs after secondary screening; the target transformation error refers to the transformation error calculated based on the optimal homography matrix.

[0110] In another exemplary embodiment of the present application, step 206 is executed to perform dynamic texture mapping in the digital twin model of the target cultural relic according to the feature matching result to realize cultural relic activation. Write a shader program, use the program algorithm to generate textures in the fragment shader, so that when the texture data is updated regularly according to event triggers in the rendering loop, the corresponding dynamic texture is generated according to the successfully matched feature points in the feature matching results, and the updated texture is bound to the digital twin model of the target cultural relic to be applied, so as to show the user the cultural relic activation effect under the current perspective. The present invention uses a cultural relic activation method that combines dynamic texture mapping and feature point matching, which can avoid the problem of deviation in the mapping of matching feature point problems, and solves the problem that the previous cultural relic activation lacks more activation effects, so that users cannot experience the culture in an immersive way.

[0111] The improvements in the present invention mainly lie in feature point detection, feature description, feature matching and screening. These improved means are applied to the feature point detection stage, feature description stage and feature matching and screening stage in the cultural relics revitalization technology, which can improve the effect of cultural relics revitalization. Specifically, a multi-scale feature point detection method is adopted to ensure that feature points can be detected at different scales. The problem that certain feature points may be ignored by conventional detection methods is solved. At the same time, by designing a feature descriptor that combines color and texture information, the unique texture and details of the surface of the cultural relics can be better described. Finally, feature point recognition and matching are used, and dynamic texture mapping is applied on this basis to make the cultural relics more fully activated. This method not only enhances the effect of cultural relics revitalization, but also provides users with a new way of viewing the exhibition, fully demonstrates the historical and cultural connotations of the cultural relics, and enhances the audience's experience.

[0112] In order to verify the effectiveness of the cultural relics activation method based on the improved feature point description method in the present invention, an ablation experiment of gradually removing multi-scale detection (hereinafter referred to as HBP-KR), multiple descriptor combination (hereinafter referred to as MB-KR) and ratio detection and RANSAC geometric detection (hereinafter referred to as MHBP) will be carried out for the method of the present invention (hereinafter referred to as MHBP-KR), and the actual effect of the method of the present invention will be verified through the test results on the OxfordVGG dataset.

[0113] Figure 3 The matching result diagram of the method of the present invention and the ablation experiment is shown. It can be seen that the method of the present invention far exceeds the method of removing multi-scale detection in the number of detected feature points, and also far exceeds the method of removing improved descriptors and filtering in feature point availability and matching degree. Figure 3 The feature point detection and matching results for different scenes. Figure 3(a) and (b) in the figure are, from top to bottom, the matching results of the method of the present invention, the matching results of the method of removing multi-scale detection, and the matching results of the method of removing multiple descriptors.

[0114] The evaluation indicators involved in the test process of the method of the present invention are n left , n right , n matches , n correct ,δ, respectively represent the number of feature points extracted from the previous frame observation image, the number of feature points extracted from the next frame observation image, the number of feature point matches and the number of correct matches, and the accuracy of feature point matching. Among them,

[0115] The statistical results are shown in Table 1 below:

[0116] Table 1 Statistical results of evaluation indicators of each method in the ablation experiment

[0117] <![CDATA[n left ]]> <![CDATA[n right ]]> <![CDATA[n matches ]]> <![CDATA[n correct ]]> δ MHBP-KR 43007 35662 434 382 88.02% HBP-KR 20118 18094 281 245 87.19% MB-KR 43007 35662 587 501 85.35% MHBP 43007 35662 43007 14353 33.37%

[0118] Table 1 shows the scores of various indicators of different ablation methods on OxfordVGG. The experimental results show the effectiveness of the multi-scale feature point detection and multiple descriptor methods of the present invention.

[0119] The present application also provides an application scenario, which applies the above-mentioned cultural relics activation method based on the improved feature point description method. Specifically: The cultural relics activation method based on the improved feature point description method provided in this embodiment can be applied in the digital twin cultural relics activation scenario. The scenario includes data acquisition, data processing and modeling, feature point detection and recognition, feature description and matching, dynamic texture mapping, interactive display and user experience, etc. The method of the present invention belongs to the feature point detection and recognition and feature description and matching links.

[0120] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 4As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the result data of feature point recognition, feature description, feature matching and screening, and cultural relics activation processing data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a cultural relics activation method based on an improved feature point description method is implemented.

[0121] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components. In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0122] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0123] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0124] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0125] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0126] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0127] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0128] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for revitalizing cultural relics based on an improved feature point description method, characterized in that: The cultural relics activation method based on the improved feature point description method includes: Obtain the observation image of the digital twin model of the target cultural relic at the current observation angle; For the currently observed image, using a multi-scale feature point detection method to detect feature points at different scales in the currently observed image; The improved feature point description method is used to describe the feature points at each scale in the current observed image, and an improved feature descriptor corresponding to the feature points at each scale is obtained; the improved feature point description method is based on the traditional BRIEF descriptor and combines color features and texture features to form an improved feature descriptor; Perform feature matching and screening according to the improved feature descriptors corresponding to the feature points at each scale in the current observation image and the previous observation image, to obtain a feature matching result; According to the feature matching results, dynamic texture mapping is performed in the digital twin model of the target cultural relic to realize cultural relic activation.

2. The cultural relics activation method based on the improved feature point description method according to claim 1 is characterized in that: For the currently observed image, a multi-scale feature point detection method is used to detect feature points at different scales in the currently observed image, specifically including: The currently observed image is scaled according to different scaling ratios to obtain an image pyramid corresponding to the currently observed image; each layer of the image pyramid corresponds to a scaled image of a scaling ratio; each layer of the image pyramid corresponds to a scale; and one observed image corresponds to multiple scaled images of different scales; Detect feature points in each zoomed image corresponding to the current observed image.

3. The cultural relics activation method based on the improved feature point description method according to claim 2 is characterized in that: Detecting feature points in each zoomed image corresponding to the current observed image specifically includes: For each zoomed image, the pixel gradient of each pixel point in a preset sliding window in the zoomed image in the x direction and the y direction is calculated; The autocorrelation matrix of each pixel point is calculated according to the pixel gradient of each pixel point in the x direction and the y direction in the preset sliding window; wherein, in C=I x I y ; M(x,y) represents the autocorrelation matrix at the pixel point (x,y), I x Represents the pixel gradient of the pixel in the x direction, I y Represents the pixel gradient of the pixel in the y direction; Calculate the Harris response value of each pixel in the preset sliding window according to the autocorrelation matrix; From the pixels whose Harris response values ​​are greater than a preset threshold, the pixels whose Harris response values ​​are local maximum values ​​are selected as feature points in the scaled image.

4. The cultural relics activation method based on the improved feature point description method according to claim 3 is characterized in that: The improved feature point description method is used to describe the feature points at each scale in the current observed image, and an improved feature descriptor corresponding to the feature points at each scale is obtained, specifically including: For each feature point at each scale in the currently observed image, a color histogram in a first preset area around the feature point is calculated; the first preset area includes the corresponding feature point; the color histogram is used to reflect the color characteristics of the feature point; For each feature point at each scale in the currently observed image, the LBP value in a second preset area around the feature point is calculated; the second preset area includes the corresponding feature point; the LBP value is used to reflect the texture feature of the feature point; For each feature point at each scale in the currently observed image, calculate the BRIEF feature descriptor in a third preset area around the feature point; the third preset area includes the corresponding feature point; For the feature points at each scale in the currently observed image, the color histogram, LBP value and BRIEF feature descriptor of the feature points are combined to obtain an improved feature descriptor corresponding to the feature points.

5. The cultural relics activation method based on the improved feature point description method according to claim 4 is characterized in that: Feature matching and screening are performed based on the improved feature descriptors corresponding to the feature points at each scale in the current observation image and the previous observation image to obtain feature matching results, specifically including: Restoring the feature points at each scale corresponding to the current observed image to the coordinate system of the original image to obtain a number of restored feature points; the original image refers to the observed image before scaling; For two adjacent frames of observation images, a preliminary matching point of each restored feature point in the current observation image is determined by using a KNN matching method; the preliminary matching point refers to the restored feature points in the previous observation image that are closest to the restored feature points in the current observation image; Each restored feature point in the current observed image and the corresponding K preliminary matching points constitute K preliminary matching point pairs; the K preliminary matching point pairs include target preliminary matching point pairs and non-target preliminary matching point pairs; the target preliminary matching point pair refers to the preliminary matching point pair with the smallest distance between the restored feature point and the preliminary matching point; the non-target preliminary matching point pair refers to the matching point pair in the preliminary matching point pairs excluding the target preliminary matching point pair; According to the distance between the matching points in the target preliminary matching point pair and the distance between the matching points in each of the non-target preliminary matching point pairs, a ratio detection method is applied to screen the target preliminary matching point pairs corresponding to the restored feature points in the current observation image, and obtain the preliminary screened matching point pairs corresponding to each restored feature point in the current observation image; one restored feature point in the current observation image corresponds to one preliminary screened matching point pair; The RANSAC method is used to screen the preliminary screening matching point pairs corresponding to all restored feature points in the current observed image to obtain the matching point pairs after secondary screening; the matching point pairs after secondary screening are the feature matching results.

6. The cultural relics activation method based on the improved feature point description method according to claim 5 is characterized in that: The expression of the ratio detection method is: good matches={X∣X,Y j ∈matches,d(X)<ratio×d(Y1),d(X)<ratio×d(Y2),...,d(X)<ratio×d(Y j )} Among them, good matches represents the set list of preliminary screened matching point pairs; matches represents the set list of K preliminary matching point pairs; X represents the target preliminary matching point pair; Y j represents the jth non-target preliminary matching point pair; ratio represents the scaling factor; d(X) represents the distance between the matching points in the target preliminary matching point pair; d(Y j ) represents the distance between the matching points in the jth non-target preliminary matching point pair.

7. The method for revitalizing cultural relics based on the improved feature point description method according to claim 5, characterized in that: The RANSAC method is used to screen the preliminary screening matching point pairs corresponding to all restored feature points in the current observed image, and the matching point pairs after secondary screening are obtained, which specifically include: Randomly extracting a plurality of matching point pairs from all the initially screened matching point pairs corresponding to all restored feature points in the current observed image; Using the extracted matching point pairs to estimate a homography matrix; the homography matrix is ​​used to represent the relationship between matching points in the extracted matching point pairs; Calculate the transformation error for each preliminary screening matching point pair using the currently estimated homography matrix, and obtain the transformation error corresponding to each preliminary screening matching point pair; Randomly extract multiple matching point pairs from the preliminary screened matching point pairs whose transformation errors are less than the preset transformation error threshold, and return to the step of "estimating the homography matrix using the extracted matching point pairs"; When the number of iterations reaches the preset number of iterations, the optimal homography matrix is ​​obtained, or when the homography matrix calculated in the next iteration process is smaller than the homography matrix of the previous iteration process, the homography matrix of the previous iteration process is regarded as the optimal homography matrix; Use the optimal homography matrix to calculate the transformation error corresponding to each pair of preliminary screened matching points; Preliminary screening matching point pairs whose target transformation error is less than a preset transformation error threshold are screened out to obtain matching point pairs after secondary screening; the target transformation error refers to the transformation error calculated based on the optimal homography matrix.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for revitalizing cultural relics based on the improved feature point description method described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for revitalizing cultural relics based on an improved feature point description method described in any one of claims 1 to 7 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for revitalizing cultural relics based on an improved feature point description method described in any one of claims 1 to 7 is implemented.

Citation Information

Patent Citations

  • Cultural relic surface texture feature extraction method based on cellular automaton

    CN107194930A

  • Har-SURF-RAN feature point matching method for stereoscopic visual three-dimensional reconstruction

    CN108416801A

  • Improved ORB algorithm based on fusion of improved LBP features and LNDP features

    CN110738265A

  • ORB image matching method fusing color and scale features

    CN115187791A

  • Visual SLAM-based cultural relic display augmented reality method

    CN115346054A