Non-missing digital display method based on virtual reality

Through the preprocessing and three-dimensional model establishment of intangible cultural heritage performance videos, combined with support vector machine and IMU technology, the problems of insufficient video jitter and interaction are solved, the user experience and hardware efficiency are improved, and high-quality digital display of intangible cultural heritage is achieved.

CN119027581AInactive Publication Date: 2024-11-26XUZHOU COLLEGE OF INDAL TECH
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411002826.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-25
Publication Date
2024-11-26
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

When collecting intangible cultural heritage performance videos, it is easy to experience tremor, resulting in unstable video images, making it difficult to accurately establish a three-dimensional performance model, and lack of interaction between users and performers, affecting the user experience.

Method used

Preprocessing is performed through the video acquisition device, jitter is eliminated, a three-dimensional performance model is established, and the support vector machine is used to identify interactive actions, and the VR glasses position is calculated in combination with the IMU to achieve accurate projection of the three-dimensional model and user interaction prompts.

Benefits of technology

Improve video quality, ensure the accuracy of the three-dimensional model, enhance user experience, reduce hardware costs, and improve operation and maintenance convenience and expansion.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027581A_ABST
    Figure CN119027581A_ABST
Patent Text Reader

Abstract

The invention discloses a non-missing digital display method based on virtual reality. S1, video acquisition; performing video acquisition through video acquisition equipment; s2, preprocessing the non-missing performance video, and uploading the video to a cloud server; preprocessing is carried out during non-immature performance video collection, and the problem that the non-immature performance culture display effect is poor due to video jitter is solved; s3, establishing a three-dimensional performance model; extracting the coordinate and attitude information of the non-missing performers in the preprocessed non-missing performance video from the cloud server, and establishing a three-dimensional performance model; s4, performing interaction scene voice prompt; and S5, performing non-missing display. According to the method and the device, the collected non-perpetual performance video is preprocessed, unplanned jumping of a video picture caused by jitter and accidental touch is eliminated, and the quality of original data of a virtual reality technology is improved, so that the quality of user experience is improved, and the hardware and space cost of local operation is reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of virtual reality, and in particular relates to a method for digital display of intangible cultural heritage based on virtual reality. Background Art

[0002] The characteristics of the intangible cultural heritage performances of intangible cultural heritage performers are mainly reflected in their liveliness, artistic and cultural educational value, practicality, life, and ethnic and regional characteristics. Intangible cultural heritage performances are very popular with the public, but when collecting and broadcasting videos of intangible cultural heritage performances on site, the following problems are prone to occur:

[0003] 1. When collecting the intangible cultural heritage performance videos of intangible cultural heritage performers, the videos may shake, resulting in unstable and poorly coherent intangible cultural heritage performance videos, and inaccurate subsequent generation of 3D performance models, which gives a poor user experience;

[0004] 2. It is difficult to determine the position of VR glasses by extracting feature points, which causes deviation in the display position of intangible cultural heritage and affects the user experience;

[0005] Traditional virtual reality display technology lacks the necessary interaction between users and performers, making it difficult for users to fully immerse themselves in the performance.

[0006] Therefore, in response to the above technical problems, it is necessary to provide a digital display method of intangible cultural heritage based on virtual reality.

[0007] The information disclosed in this background technology section is only intended to enhance the understanding of the overall background of the invention and should not be regarded as an acknowledgment or any form of suggestion that the information constitutes the prior art already known to a person skilled in the art. Summary of the invention

[0008] The purpose of the present invention is to provide a method for digital display of intangible cultural heritage based on virtual reality, which can solve the above problems.

[0009] In order to achieve the above object, a technical solution provided by a specific embodiment of the present invention is as follows:

[0010] A method for digital display of intangible cultural heritage based on virtual reality, comprising the following steps;

[0011] S1. Video acquisition: Video acquisition is performed through a video acquisition device;

[0012] S2. Preprocess the intangible cultural heritage performance video and upload it to the cloud server; preprocess the intangible cultural heritage performance video when it is collected to eliminate the problem of poor intangible cultural heritage display effect caused by video jitter;

[0013] S3, establishing a three-dimensional performance model; extracting the coordinates and posture information of the intangible cultural heritage performers in the pre-processed intangible cultural heritage performance video on the cloud server to establish a three-dimensional performance model;

[0014] S4, voice prompts for performance interaction scenarios: Constructing a performance interaction model based on support vector machine, identifying the interaction scenarios in the intangible cultural heritage performance process, and providing advance voice prompts for the performance interaction scenarios;

[0015] The data obtained from the 3D performance model is used as input samples frame by frame, and combined with other interactive images to make the total sample data volume 500-800 pictures. The support vector machine is a model for solving binary classification problems. The performance interaction model and objective function are built based on the support vector machine, which can transform the complex performance interaction recognition problem into a binary classification problem and improve the calculation efficiency.

[0016] S5, intangible cultural heritage display; pre-process the acceleration and angular velocity information collected by the IMU on the VR glasses, calculate the position coordinates of the VR glasses, and if the difference between the VR glasses position and the specified display position is less than a threshold, project the three-dimensional performance model onto the VR glasses for display; otherwise, prompt the user to move to experience the intangible cultural heritage display.

[0017] In one or more embodiments of the present invention, the S2 pre-processing comprises the following steps:

[0018] S21, set the intangible cultural heritage performance video frame I n and I n-1 Corresponding pixel coordinate X n and X n-1 satisfy

[0019] X n =C n X n-1 +B n

[0020] Among them C n To select the threshold, B n is the translation threshold, and the first frame of the intangible cultural heritage performance video frame is taken as the reference frame.

[0021]

[0022] S22, i and j are the numbers of the video frames of the intangible cultural heritage performance. The global transformation parameters of the nth frame image after eliminating jitter and the reference frame are and The compensation parameters after and The specific calculation formula is shown as follows:

[0023]

[0024] Among them, β is the compensation coefficient, γ is the transformation coefficient;

[0025] S23, after compensation, the n-th frame image can be transformed according to the following formula to obtain a stable video sequence, and the specific calculation formula is shown in the following formula:

[0026]

[0027] In one or more embodiments of the present invention, establishing the three-dimensional performance model includes the following steps:

[0028] S31, splitting the intangible cultural heritage performance video frame by frame according to the acquisition frequency of the intangible cultural heritage video to obtain a performance video sequence frame;

[0029] S32, preset an interval threshold, extract the intangible cultural heritage performance video sequence frames to obtain extracted video sequence frames, avoid processing all video frames, so as to reduce the amount of calculation;

[0030] S33, extracting feature points from the extracted video sequence frames to obtain extracted feature point data;

[0031] S34, matching the extracted feature points by an ICP matching algorithm to obtain the same extracted feature point data of adjacent extracted video sequence frames;

[0032] S35, establishing a scene coordinate system with the acquisition device as the origin, and obtaining the position and angle of the non-legacy performer in the adjacent extracted video sequence frames through SVD settlement;

[0033] S36, calculating the three-dimensional coordinates of the same extracted feature point data of adjacent extracted video sequence frames by using a metrology algorithm based on the positions and angles of the intangible cultural heritage performers in adjacent extracted video sequence frames;

[0034] S37. Finally, a three-dimensional performance model of the intangible cultural heritage performer during the performance is generated based on the three-dimensional coordinate data of the same extracted feature point data of adjacent extracted video sequence frames.

[0035] In one or more embodiments of the present invention, performing interactive scenario voice prompts in step S4 specifically includes the following steps:

[0036] S41. Determine whether the performance action is an interactive action by using the performance interaction model:

[0037] S42, setting the ratio of the training set to the test set, randomly selecting performance interaction pictures and ordinary pictures as training samples for training, and the remaining sample data as test samples, and determining the penalty coefficient C in the support vector machine;

[0038] S43. When continuous interactive actions are recognized, the VR glasses will give voice prompts 1-6 seconds in advance to improve the user experience.

[0039] In one or more embodiments of the present invention, the ratio of the training set to the test set is 5:5.

[0040] In one or more embodiments of the present invention, the step S4 of converting the complex performance interaction recognition problem into a binary classification problem includes the following steps:

[0041] S41. The plane equation of the classification line is:

[0042] ω*x+v=0

[0043] ω is the weight coefficient, v is the bias value, and x is the image feature;

[0044] S42. Further introduction of relaxation variable ξ i And the penalty coefficient C satisfies the following formula:

[0045] y i (ω*x i +v)≥1-ξ i

[0046] y is the sample category function. There are only two results for the binary classification problem of whether the current frame is an interactive scene;

[0047] S43, the objective function of the performance interaction model is shown as follows:

[0048]

[0049] Where q represents the total number of samples;

[0050] S44. Convert the problem into a convex quadratic programming optimization problem. The specific calculation formula is as follows:

[0051]

[0052] Where a represents the Lagrange multiplier and O represents the adaptive optimization parameter.

[0053] In one or more embodiments of the present invention, the IMU is an inertial measurement unit, which includes an accelerometer and a gyroscope. The accelerometer detects acceleration signals of three axes, and the gyroscope detects angular velocity signals.

[0054] In one or more embodiments of the present invention, the IMU is located in the middle position above the VR glasses, and the IMU is used to calculate the position of the VR glasses.

[0055] In one or more embodiments of the present invention, the IMU measures angular velocity and acceleration information. The specific form is shown in the following formula:

[0056]

[0057] in, and Represents the IMU's measured rotational angular velocity and acceleration, and are the random walks of angular velocity and acceleration at time ti, and are the white noise of angular velocity and acceleration at time ti, w ti and a ti are the true values ​​of angular velocity and acceleration, It is the transformation matrix from the information measured in the VR glasses coordinate system at time ti to the W world coordinate system, and g is the gravity constant.

[0058] In one or more embodiments of the present invention, the intangible cultural heritage display in step S5 includes the following steps:

[0059] S51, integrating the IMU information, which respectively represents the changing relationship among the speed, acceleration and rotation direction of the VR glasses. The specific formula is as follows;

[0060]

[0061] Where v, p, and R represent the speed, position, and rotation information of the VR glasses, respectively. ti+Δti It refers to the speed change from time ti to Δti, p ti+Δti It refers to the change in position from time ti to Δti, R ti+Δti It refers to the rotation change during the transformation time from time ti to Δti;

[0062] S52. After the integration processing of the IMU, the position coordinate p of the VR glasses can be obtained, where p = {x, y, z}, x represents the coordinate value of the VR glasses in the x-axis direction, y represents the coordinate value of the VR glasses in the y-axis direction, and z represents the coordinate value of the VR glasses in the z-axis direction;

[0063] Initialize the specified placement p m ={x m ,y m ,z m}, initialize the threshold α, calculate the difference between the VR glasses position and the specified display position,

[0064] if The three-dimensional performance model is projected onto VR glasses for display;

[0065] if Users are prompted to move through voice to experience the intangible cultural heritage display.

[0066] Compared with the prior art, the virtual reality-based intangible cultural heritage digital display method of the present invention has the following benefits:

[0067] 1. Pre-process the collected intangible cultural heritage performance videos to eliminate unplanned jitters caused by jitter or accidental touches, improve the quality of the original data of virtual reality technology, and thus improve the quality of user experience;

[0068] 2. Establishing a 3D performance model on a cloud server. Compared with establishing a 3D performance model locally, this solution can not only reduce the hardware and space costs of local operation and improve the convenience of operation and maintenance personnel in handling methods and codes, but also has stronger scalability, which facilitates the subsequent deployment of better 3D modeling processing algorithms on the cloud.

[0069] 3. Use IMU to calculate the position of VR glasses, and decide whether to display the three-dimensional performance model based on position judgment. If the user's position deviates greatly from the predetermined display position, it will not be displayed and the user will be reminded to move, thereby further interacting with the user and improving the digital display effect of intangible cultural heritage.

[0070] 4. Use support vector machines to build a performance interaction model, identify the interactive actions of intangible cultural heritage performers, and remind users in a timely manner, which can help users better interact with intangible cultural heritage performers and improve the user's viewing experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0072] Figure 1 This is a flow chart of a method for digital display of intangible cultural heritage based on virtual reality in one embodiment of the present invention. DETAILED DESCRIPTION

[0073] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0074] like Figure 1As shown, a method for digital display of intangible cultural heritage based on virtual reality in one embodiment of the present invention comprises the following steps:

[0075] S1: Video acquisition: Professional video acquisition equipment is used to collect the intangible cultural heritage performance videos of intangible cultural heritage performers;

[0076] When intangible cultural heritage performers perform intangible cultural heritage, video data of the performance of the intangible cultural heritage performers can be obtained by using professional video acquisition equipment to capture video.

[0077] S2: Pre-process the intangible cultural heritage performance video and upload it to the cloud server;

[0078] When professional video capture equipment is used to capture the intangible cultural heritage performance videos of intangible cultural heritage performers, jitters may occur. The reasons for these jitters are varied, such as uneven ground depressions or convexities, sudden changes in the body position of the video capturer, collisions with foreign objects, etc. These jitters will cause the generated intangible cultural heritage performance videos to be unstable and have poor continuity, and will be inaccurate when subsequently generating three-dimensional performance models, giving a poor user experience. To solve the above problems, preprocessing is performed during the collection of intangible cultural heritage performance videos to eliminate the problem of poor intangible cultural heritage display effects caused by video jitters, and to improve the quality of the shooting process of the digital display of intangible cultural heritage. The specific implementation steps are as follows:

[0079] Suppose that the video frame of the intangible cultural heritage performance is n and I n-1 Corresponding pixel coordinate X n and X n-1 satisfy

[0080] X n =C n X n-1 +B n

[0081] Among them, C n To select the threshold, B n is the translation threshold. Taking the first frame of the intangible cultural heritage performance video as the reference frame,

[0082]

[0083] Among them, i and j are the numbers of the intangible cultural heritage performance video frames, and the global transformation parameters of the nth frame image after eliminating jitter and the reference frame are and The compensation parameters after and The specific calculation formula is shown as follows:

[0084]

[0085] Among them, β is the compensation coefficient and γ is the transformation coefficient.

[0086] Therefore, after compensation, the nth frame image can be transformed according to the following formula to obtain a stable video sequence. The specific calculation formula is shown in the following formula:

[0087]

[0088] Upload the pre-processed intangible cultural heritage performance video to the cloud server.

[0089] S3: extracting coordinates and posture information of the intangible cultural heritage performers from the pre-processed intangible cultural heritage performance video on the cloud server, and establishing a three-dimensional performance model;

[0090] Compared with establishing a 3D performance model locally, establishing a 3D performance model on a cloud server can not only reduce the hardware and space costs of local operation and improve the convenience of operation and maintenance personnel in handling methods and codes, but also has stronger scalability, which facilitates the subsequent deployment of better 3D modeling processing algorithms on the cloud.

[0091] The coordinate and posture information of the intangible cultural heritage performers in the preprocessed intangible cultural heritage performance video is extracted on the cloud server, and the coordinate information and body posture information of the intangible cultural heritage performers during the performance are obtained based on the intangible cultural heritage performance video, including: the intangible cultural heritage performance video is used to perform three-dimensional modeling processing of the intangible cultural heritage performers by professional users during the performance, so as to obtain a three-dimensional performance model of the intangible cultural heritage performers during the performance.

[0092] The process of building a 3D performance model is as follows:

[0093] S31, first split the intangible cultural heritage performance video frame by frame according to the acquisition frequency of the intangible cultural heritage video to obtain a performance video sequence frame;

[0094] S32, preset an interval threshold, extract the intangible cultural heritage performance video sequence frames to obtain extracted video sequence frames, avoid processing all video frames, so as to reduce the amount of calculation, thereby further reducing the calculation cost and improving the processing efficiency;

[0095] S33, extracting feature points from the extracted video sequence frames to obtain extracted feature point data;

[0096] S34, matching the extracted feature points by an ICP matching algorithm to obtain the same extracted feature point data of adjacent extracted video sequence frames;

[0097] S35, establishing a scene coordinate system with the professional video acquisition device as the origin, and obtaining the position and angle of the non-legacy performer in the adjacent extracted video sequence frames through SVD settlement;

[0098] S36, calculating the three-dimensional coordinates of the same extracted feature point data of adjacent extracted video sequence frames by using a metrology algorithm based on the positions and angles of the intangible cultural heritage performers in adjacent extracted video sequence frames;

[0099] S37. Finally, a three-dimensional performance model of the intangible cultural heritage performer during the performance is generated based on the three-dimensional coordinate data of the same extracted feature point data of adjacent extracted video sequence frames.

[0100] S4: Construct a performance interaction model based on support vector machine to identify the interaction scenarios in the intangible cultural heritage performance process and provide advance voice prompts for the performance interaction scenarios.

[0101] The data obtained by the three-dimensional performance model is used as input samples frame by frame, and combined with other interactive images to make the total sample data volume 500-800 pictures, wherein the total number of pictures is preferably 600;

[0102] Support vector machine is a model for solving binary classification problems. Constructing the performance interaction model and objective function based on support vector machine can transform the complex performance interaction recognition problem into a binary classification problem and improve computational efficiency.

[0103] Let the plane equation of the classification line be:

[0104] ω*x+v=0

[0105] Among them, ω is the weight coefficient, v is the bias value, and x is the image feature;

[0106] Further introduce the slack variable ξ i And the penalty coefficient C satisfies the following formula:

[0107] y i (ω*x i +v)≥1-ξ i

[0108] Among them, y is the sample category function. There are only two results for the binary classification problem of whether the current frame is an interactive scene;

[0109] From this, the objective function of the performance interaction model can be obtained as follows:

[0110]

[0111] Where q represents the total number of samples;

[0112] Thus, the problem is converted into a convex quadratic programming optimization problem. The specific calculation formula is shown as follows:

[0113]

[0114] Where a represents the Lagrange multiplier and O represents the adaptive optimization parameter.

[0115] The performance interaction model is used to determine whether the performance action is an interactive action: the ratio of the training set to the test set is set to 5:5, and performance interaction pictures and ordinary pictures are randomly selected as training samples for training. The remaining sample data are used as test samples to determine the penalty coefficient C in the support vector machine.

[0116] When continuous interactive actions are recognized, the VR glasses will give voice prompts 1-6 seconds in advance to improve the user experience. Preferably, the VR glasses will give voice prompts 3 seconds in advance.

[0117] S5: Preprocess the acceleration and angular velocity information collected by the IMU on the VR glasses, calculate the position coordinates of the VR glasses, and if the difference between the VR glasses position and the specified display position is less than a threshold, project the three-dimensional performance model onto the VR glasses for display; otherwise, prompt the user to move to experience the intangible cultural heritage display.

[0118] IMU stands for inertial measurement unit, which consists of an accelerometer and a gyroscope. The accelerometer detects three-axis acceleration signals, and the gyroscope detects angular velocity signals.

[0119] The IMU in this application is located in the middle of the entire top of the VR glasses and is used to calculate the position of the VR glasses. The IMU can mainly measure angular velocity and acceleration information. The specific form is shown in the following formula:

[0120]

[0121] in, and Represents the IMU's measured rotational angular velocity and acceleration, and are the random walks of angular velocity and acceleration at time ti, and are the white noise of angular velocity and acceleration at time ti, w ti and a ti are the true values ​​of angular velocity and acceleration, It is the transformation matrix from the information measured in the VR glasses coordinate system at time ti to the W world coordinate system, and g is the gravity constant.

[0122] The IMU information is integrated. The specific formula is shown below, which represents the changing relationship among the speed, acceleration and rotation direction of the VR glasses.

[0123]

[0124] Where v, p, and R represent the speed, position, and rotation information of the VR glasses, respectively. ti+ΔtiIt refers to the speed change during the transformation time from time ti to Δti, and the same applies to the others.

[0125] After the IMU integration processing, the VR glasses position coordinates p can be obtained, where p = {x, y, z}, x represents the coordinate value of the VR glasses in the x-axis direction, and y represents the coordinate value of the VR glasses in the y-axis direction.

[0126] z represents the coordinate value of the VR glasses in the z-axis direction.

[0127] Initialize the specified placement p m ={x m ,y m ,z m}, initialize the threshold α, calculate the difference between the VR glasses position and the specified display position, if The 3D performance model is projected onto VR glasses for display. Users are prompted to move through voice to experience the intangible cultural heritage display.

[0128] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

[0129] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A digital display method of intangible cultural heritage based on virtual reality, It is characterized in that The method comprises the following steps: S1. Video acquisition: Video acquisition is performed through a video acquisition device; S2. Preprocess the intangible cultural heritage performance video and upload it to the cloud server; preprocess the intangible cultural heritage performance video when it is collected to eliminate the problem of poor intangible cultural heritage display effect caused by video jitter; S3, establishing a three-dimensional performance model; extracting the coordinates and posture information of the intangible cultural heritage performers in the pre-processed intangible cultural heritage performance video on the cloud server to establish a three-dimensional performance model; S4, performance interactive scenario voice prompts; A performance interaction model is built based on support vector machines to identify the interaction scenarios in the intangible cultural heritage performance process and provide advance voice prompts for the performance interaction scenarios. The data obtained from the 3D performance model is used as input samples frame by frame, and combined with other interactive images to make the total sample data volume 500-800 pictures. The support vector machine is a model for solving binary classification problems. The performance interaction model and objective function are built based on the support vector machine, which can transform the complex performance interaction recognition problem into a binary classification problem and improve the calculation efficiency. S5, intangible cultural heritage display; pre-process the acceleration and angular velocity information collected by the IMU on the VR glasses, calculate the position coordinates of the VR glasses, and if the difference between the VR glasses position and the specified display position is less than a threshold, project the three-dimensional performance model onto the VR glasses for display; otherwise, prompt the user to move to experience the intangible cultural heritage display.

2. According to the method for digital display of intangible cultural heritage based on virtual reality according to claim 1, It is characterized in that The S2 pretreatment comprises the following steps: S21, set the intangible cultural heritage performance video frame I n and I n-1 Corresponding pixel coordinate X n and X n-1 satisfy X n =C n X n-1 +B n Among them C n To select the threshold, B n is the translation threshold, and the first frame of the intangible cultural heritage performance video frame is taken as the reference frame. S22, i and j are the numbers of the video frames of the intangible cultural heritage performance. The global transformation parameters of the nth frame image after eliminating jitter and the reference frame are and The compensation parameters after and The specific calculation formula is shown as follows: Among them, β is the compensation coefficient, γ is the transformation coefficient; S23, after compensation, the n-th frame image can be transformed according to the following formula to obtain a stable video sequence, and the specific calculation formula is shown in the following formula:

3. According to the virtual reality-based intangible cultural heritage digital display method of claim 1, It is characterized in that Establishing the three-dimensional performance model includes the following steps: S31, splitting the intangible cultural heritage performance video frame by frame according to the acquisition frequency of the intangible cultural heritage video to obtain a performance video sequence frame; S32, preset an interval threshold, extract the intangible cultural heritage performance video sequence frames to obtain extracted video sequence frames, avoid processing all video frames, so as to reduce the amount of calculation; S33, extracting feature points from the extracted video sequence frames to obtain extracted feature point data; S34, matching the extracted feature points by an ICP matching algorithm to obtain the same extracted feature point data of adjacent extracted video sequence frames; S35, establishing a scene coordinate system with the acquisition device as the origin, and obtaining the position and angle of the non-legacy performer in the adjacent extracted video sequence frames through SVD settlement; S36, calculating the three-dimensional coordinates of the same extracted feature point data of adjacent extracted video sequence frames by using a metrology algorithm based on the positions and angles of the intangible cultural heritage performers in adjacent extracted video sequence frames; S37. Finally, a three-dimensional performance model of the intangible cultural heritage performer during the performance is generated based on the three-dimensional coordinate data of the same extracted feature point data of adjacent extracted video sequence frames.

4. According to the virtual reality-based intangible cultural heritage digital display method of claim 1, It is characterized in that The performing interactive scenario voice prompt in step S4 specifically includes the following steps: S41. Determine whether the performance action is an interactive action by using the performance interaction model: S42, setting the ratio of the training set to the test set, randomly selecting performance interaction pictures and ordinary pictures as training samples for training, and the remaining sample data as test samples, and determining the penalty coefficient C in the support vector machine; S43. When continuous interactive actions are recognized, the VR glasses will give voice prompts 1-6 seconds in advance to improve the user experience.

5. According to the method for digital display of intangible cultural heritage based on virtual reality according to claim 4, It is characterized in that The ratio of the training set to the test set is 5:

5.

6. According to the virtual reality-based intangible cultural heritage digital display method of claim 5, It is characterized in that The step S4 converts the complex performance interaction recognition problem into a binary classification problem, including the following steps: S41. The plane equation of the classification line is: ω*x+v=0 ω is the weight coefficient, v is the bias value, and x is the image feature; S42. Further introduction of slack variable ξ i And the penalty coefficient C satisfies the following formula: y i (ω*x i +v)≥1-ξ i y is the sample category function. There are only two results for the binary classification problem of whether the current frame is an interactive scene; S43, the objective function of the performance interaction model is shown as follows: Where q represents the total number of samples; S44. Convert the problem into a convex quadratic programming optimization problem. The specific calculation formula is as follows: Where a represents the Lagrange multiplier and O represents the adaptive optimization parameter.

7. The method for digital display of intangible cultural heritage based on virtual reality according to claim 1, It is characterized in that The IMU is an inertial measurement unit, which includes an accelerometer and a gyroscope. The accelerometer detects acceleration signals of three axes, and the gyroscope detects angular velocity signals.

8. The method for digital display of intangible cultural heritage based on virtual reality according to claim 1, It is characterized in that The IMU is located in the middle position above the VR glasses, and the IMU is used to calculate the position of the VR glasses.

9. The method for digital display of intangible cultural heritage based on virtual reality according to claim 1, It is characterized in that The IMU measures angular velocity and acceleration information, which is shown in the following formula: in, and Represents the IMU's measured rotational angular velocity and acceleration, and are the random walks of angular velocity and acceleration at time ti, and are the white noise of angular velocity and acceleration at time ti, w ti and a ti are the true values ​​of angular velocity and acceleration, It is the transformation matrix from the information measured in the VR glasses coordinate system at time ti to the W world coordinate system, and g is the gravity constant.

10. The method for digital display of intangible cultural heritage based on virtual reality according to claim 9, It is characterized in that The intangible cultural heritage display in step S5 includes the following steps: S51, integrating the IMU information, which respectively represents the changing relationship among the speed, acceleration and rotation direction of the VR glasses. The specific formula is as follows; Where v, p, and R represent the speed, position, and rotation information of the VR glasses, respectively. ti+Δti It refers to the speed change from time ti to Δti, p ti+Δti It refers to the change in position from time ti to Δti, R ti+Δti It refers to the rotation change during the transformation time from time ti to Δti; S52. After the integration processing of the IMU, the position coordinate p of the VR glasses can be obtained, where p = {x, y, z}, x represents the coordinate value of the VR glasses in the x-axis direction, y represents the coordinate value of the VR glasses in the y-axis direction, and z represents the coordinate value of the VR glasses in the z-axis direction; Initialize the specified placement p m ={x m ,y m ,z m }, initialize the threshold α, calculate the difference between the VR glasses position and the specified display position, if The three-dimensional performance model is projected onto VR glasses for display; if Users are prompted to move through voice to experience the intangible cultural heritage display.

Citation Information

Cited By

  • Non-abandoned inheritance display method and system based on artificial intelligence

    CN119672198A

  • Non-abandoned artwork virtual display method and system based on VR technology

    CN119992893A