Non-perpetual digital image science popularization interaction system based on augmented reality technology

Abstract

The invention discloses a non-perpetual digital image science popularization interaction system based on an augmented reality technology, and the system comprises a non-perpetual digital resource collection and management module which is used for collecting, editing and storing the digital image content related to intangible cultural heritage; the augmented reality scene recognition module is used for recognizing a reality scene and acquiring related information; the content retrieval and recommendation module is used for retrieving and pushing non-perpetual digital image content related to the current scene of the user; the augmented reality rendering display module is used for superposing and displaying the non-perpetual digital image content on the user terminal equipment; the multi-mode interaction module is used for providing multiple interaction modes and obtaining interaction operation information of a user; the content dynamic response module is used for performing response and content updating on the non-perpetual digital image content; and the learning data acquisition and feedback module is used for acquiring user learning data and interaction behaviors in real time and generating personalized learning feedback. According to the method, the interactivity, immersion and individuation level of non-peoples science popularization propagation are improved.

Description

Technical Field

[0001] The present invention relates to the fields of augmented reality technology, digital image processing and science popularization interaction, and in particular to an intangible cultural heritage digital image science popularization interaction system based on augmented reality technology, which is suitable for applications such as digital collection of intangible cultural heritage content, AR scene recognition, virtual-reality fusion display, multimodal interaction and personalized knowledge push. Background Art

[0002] Currently, the protection and dissemination of intangible cultural heritage (ICH) has become a focus of widespread societal attention. With the continuous advancement of digital technology, an increasing number of ICH items are being recorded and displayed digitally, providing new avenues for the inheritance and popularization of ICH. Existing digital ICH popularization efforts primarily focus on the collection and display of static resources such as images, audio and video, and 3D models, typically distributing relevant content through platforms such as websites, mobile applications, or digital exhibition halls. While these digital display methods have played a role in enriching ICH expression, they still suffer from significant deficiencies in user experience, interactivity, and immersion. Users are often limited to passively browsing content, struggling to engage deeply and experience it immersively, which in turn hinders the dissemination of ICH knowledge and a deeper understanding of its cultural connotations.

[0003] In recent years, augmented reality (AR) technology has been increasingly used in education, museums, tourism and other fields. Some cultural venues and museums have tried to combine AR with intangible cultural heritage content. This type of AR application usually scans physical objects, images or specific scenes, and superimposes virtual content such as three-dimensional models, animations or multimedia explanations on the real scene to achieve a display effect that combines the virtual and the real. However, existing AR displays are mostly centered on a single scene or a specific exhibit, mainly realizing the simple superposition of a single content. There is a lack of systematic, standardized digital collection and unified management of intangible cultural heritage resources, which limits the integrity and scalability of the content. In addition, the current content push method is relatively simple, making it difficult to achieve intelligent and personalized intangible cultural heritage knowledge recommendations based on user interests, geographic location or actual needs. As a result, the enthusiasm of users to actively participate and the diversity of their experience are affected.

[0004] In terms of user interaction, existing technologies are mostly based on simple button clicks and page switching. Although some applications support basic gesture recognition or voice commands, the overall form of interaction is still relatively monotonous. Users can only passively receive information and lack active participation and in-depth experience. In addition, existing systems usually do not have an effective learning data collection and feedback mechanism, and are unable to track users' learning behavior and knowledge mastery in real time. It is also difficult to make personalized recommendations or organize interactive educational activities based on users' learning performance. This makes it difficult for the intangible cultural heritage science popularization system to form an effective learning incentive closed loop, and both user participation and knowledge dissemination efficiency need to be improved.

[0005] In summary, existing technologies have significant deficiencies in the collection and management of intangible cultural heritage digital content, the display of augmented reality (AR) virtual-real integration, multimodal interactive experiences, and personalized science popularization push, making it difficult to meet the current needs of intangible cultural heritage protection and inheritance, and the high-quality development of science popularization education. Therefore, there is an urgent need for a new science popularization interactive method that can achieve the digitization of intangible cultural heritage content, AR virtual-real integration, multimodal interaction, and a closed-loop learning feedback loop. This will enhance the immersiveness, fun, and personalization of intangible cultural heritage knowledge dissemination, and promote the widespread dissemination and innovative inheritance of intangible cultural heritage. Summary of the Invention

[0006] One purpose of the present invention is to propose an intangible cultural heritage digital image popular science interactive system based on augmented reality technology. The present invention makes full use of augmented reality technology, digital image processing and multimodal human-computer interaction technology, and describes in detail the implementation process of systematic digital collection of intangible cultural heritage content, augmented reality scene recognition and virtual-reality fusion display, personalized content push and learning behavior feedback. It has the advantages of strong immersion, high interactivity and good popular science communication effect.

[0007] The intangible cultural heritage digital image popular science interactive system based on augmented reality technology according to an embodiment of the present invention includes:

[0008] The intangible cultural heritage digital resource collection and management module is used to collect data related to intangible cultural heritage, edit, process and store the data in a standardized manner, and establish an intangible cultural heritage digital resource library;

[0009] The augmented reality scene recognition module is used to identify the real scene through augmented reality technology on the user terminal device and obtain specific objects, images or geographic location information;

[0010] The content retrieval and recommendation module is used to retrieve and push intangible cultural heritage digital image content related to the scene or location from the intangible cultural heritage digital resource library based on the recognition results of the real scene or the user's geographical location information, thereby realizing personalized popular science knowledge recommendations;

[0011] The augmented reality rendering display module is used to overlay the retrieved intangible cultural heritage digital image content on the display interface of the user terminal device in an augmented reality manner, achieving a display effect that combines virtual and real;

[0012] The multimodal interaction module is used to provide a multimodal interaction interface on the user terminal device and obtain the user's interactive operation information;

[0013] The content dynamic response module is used to respond to user operations based on user interaction information and dynamically update the displayed intangible cultural heritage digital image content;

[0014] The learning data collection and feedback module is used to collect users' learning data and interactive behaviors in real time, generate learning feedback information, and display learning results on user terminal devices, or push intangible cultural heritage knowledge competitions and interactive question-answering educational activities.

[0015] Optionally, modules can be connected using the following methods:

[0016] S1. Collect and process images, videos, three-dimensional models, and animation data related to intangible cultural heritage, and perform standardized editing and storage on the data to build an intangible cultural heritage digital resource library;

[0017] S2. Using augmented reality scene recognition on the user terminal device, real-time images of the user's real environment or geographic location information are collected to obtain recognition results;

[0018] S3. Based on the recognition result or geographic location information, the content retrieval and recommendation module retrieves intangible cultural heritage digital image content related to the current scene or location in the intangible cultural heritage digital resource library, and performs relevance sorting and personalized recommendation;

[0019] S4, overlaying the retrieved intangible cultural heritage digital image content onto the display interface of the user terminal device through an augmented reality rendering display module, thereby realizing a digital content display that combines virtuality and reality;

[0020] S5, the multimodal interaction module obtains multimodal user interaction operation information including gesture recognition, touch screen operation, voice command and facial expression recognition, and integrates it into interaction instructions;

[0021] S6. The interactive instruction is parsed by the content dynamic response module according to the corresponding operation type, and the displayed intangible cultural heritage digital image content is browsed, enlarged, rotated, scaled, switched, or interactive games, knowledge quizzes, and knowledge challenge functions are triggered, and the display of the content is dynamically updated;

[0022] S7. Through the learning data collection and feedback module, the user's learning progress, interactive behavior and knowledge mastery are collected in real time, analyzed and generated learning feedback information, and learning results, personalized reports or invitations to intangible cultural heritage knowledge competitions and interactive educational activities are displayed on the user's terminal device.

[0023] Optionally, step S1 specifically includes:

[0024] S11, capturing images of intangible cultural heritage objects, performances, and scenes using digital camera equipment to obtain original image data G;

[0025] S12, using audio and video acquisition equipment to capture video of the intangible cultural heritage related performance and explanation process to obtain original video data V;

[0026] S13, using a 3D scanner or modeling software to perform 3D modeling on the intangible cultural heritage object to generate 3D model data M;

[0027] S14, animating the intangible cultural heritage related content using an animation production tool to obtain animation data A;

[0028] S15. Perform format conversion, editing, denoising, and enhancement processing on the acquired image data G, video data V, 3D model data M, and animation data A to obtain standardized digital content D. The formula is:

[0029] D=f(G,V,M,A),

[0030] Wherein, f represents the comprehensive function of digital content editing and processing;

[0031] S16. Classify, annotate and index the edited digital content D according to themes, types and sources, and establish an intangible cultural heritage digital resource library for subsequent retrieval and access.

[0032] Optionally, step S2 specifically includes:

[0033] S21, start the augmented reality recognition module on the user terminal device, and use the built-in camera to collect real-time scene image stream I;

[0034] S22. Apply an image recognition algorithm to extract features from the collected scene image stream I to obtain a scene feature vector F, namely:

[0035] F=φ(I),

[0036] Here, φ represents the image feature extraction function, which can extract key scene structure features and visual descriptions through convolutional neural networks (CNN), SIFT, or SURF mainstream visual feature extraction algorithms to achieve in-depth understanding and analysis of scene content;

[0037] S23, by comparing the scene feature vector F with the preset object and image feature database F db , determine whether there is a matching target, if so, determine the recognition result R of the specific object or image obj ,The comparison process can adopt vector distance nearest neighbor search or deep learning ,classification model;

[0038] S24. Obtain geographic location information L of the user terminal device, where the geographic location information includes latitude and longitude, wherein L is obtained by a global positioning system (GPS) or other positioning module, and L = (lat, lon), where lat represents latitude and lon represents longitude;

[0039] S25, the recognition result R objCombined with geographic location information L, it serves as an input condition for subsequent retrieval and push of intangible cultural heritage digital image content, effectively improving the relevance and personalization of content to user scenarios.

[0040] Optionally, step S3 specifically includes:

[0041] S31, receiving the scene recognition result R output by the augmented reality recognition module obj and geographic location information L;

[0042] S32: Based on the scene recognition result and the geographical location information, the intangible cultural heritage digital image content C related to the scene or location information is searched in the intangible cultural heritage digital resource library. The search process is as follows:

[0043] C=Ψ(R obj ,L),

[0044] Among them, Ψ represents a resource retrieval function based on scene recognition and geographic location. It can comprehensively call keyword matching, content tag screening, and location similarity algorithms based on the user's current scene recognition results and geographic location to efficiently screen the intangible cultural heritage content in the database and recall content that is closely related to the actual scene;

[0045] S33. Sort the retrieved content C by relevance to obtain a sorted content set C. sort ,Right now:

[0046] C sort =Ω(C,R obj ,L),

[0047] Ω represents a content ranking function based on relevance and user preferences. It can perform multi-level weighted ranking based on multiple factors such as the relevance of content to the scene or location, user historical preferences, and content popularity, ensuring that the content pushed to users is the most relevant and prioritized.

[0048] S34, the sorted content set C sort Push to the user's terminal device and provide personalized scientific knowledge recommendations based on the user's current scenario or geographic location.

[0049] Optionally, step S4 specifically includes:

[0050] S41. Receive the sorted intangible cultural heritage digital image content set C on the user terminal device. sort ;

[0051] S42, using the augmented reality rendering display module to display the content set C sort Each item in C i Perform three-dimensional spatial positioning and determine the spatial coordinates P of the superimposed display i =(xi ,y i , z i ), where x i ,y i ,z i is the three-dimensional coordinate corresponding to the real scene;

[0052] S43, according to the scene image I and space coordinates P obtained in real time by the user terminal device camera i , using superposition rendering to transform the intangible cultural heritage digital image content C i It is superimposed on the real scene image I in the form of a virtual layer to generate an augmented reality display image A i ,Right now:

[0053] A i =Λ(I,C i ,P i ),

[0054] Here, Λ represents the augmented reality content overlay rendering function, which can achieve spatial projection, transparency adjustment, and dynamic interactive rendering of virtual content in real scenes through OpenGL and Unity3D three-dimensional rendering engines, ensuring that the display effect of virtual and real fusion is natural and realistic;

[0055] S44: Presenting augmented reality display image A on the display interface of the user terminal device i , allowing users to intuitively see the intangible cultural heritage digital image content that matches the spatial position of the real scene in the real environment, achieving a display effect that combines virtual and real.

[0056] Optionally, step S5 specifically includes:

[0057] S51. Initializing a multimodal interaction module on a user terminal device, wherein the multimodal interaction module supports three interaction modes: gesture recognition, touch screen operation, and voice command input;

[0058] S52, collecting user hand motion data G through gesture recognition sensor hand , and identify the gesture command S g ,Right now:

[0059] S g =α(G hand ),

[0060] Here, α represents the gesture recognition processing function, which can realize the classification and recognition of user gestures based on convolutional neural networks, time series analysis, or skeleton-based image processing algorithms;

[0061] S53, collecting user touch input data T through the touch screen touch , and identify and obtain the touch command S t, the formula is:

[0062] S t =β(T touch ),

[0063] Among them, β represents the touch input processing function, which can analyze multi-touch, sliding, clicking and other operations to achieve rich interface interactions;

[0064] S54, collecting user voice input data V through the microphone voice , and recognize the voice command S v , the formula is:

[0065] S v =γ(V voice ),

[0066] Here, γ represents the speech recognition processing function, which can call the ASR (automatic speech recognition) engine on the device or cloud to convert the user's natural voice commands into interactive commands that the system can recognize;

[0067] S55, the gesture command S g , touch command S t and voice commands S v Merge to form user interaction operation information S user ,Right now:

[0068] S user ={S g ,S t ,S v};

[0069] S56, the user interaction operation information S user Sent to the system interaction processing module as input conditions for subsequent intangible cultural heritage digital image content responses.

[0070] Optionally, step S6 specifically includes:

[0071] S61, receiving user interaction operation information S user , including gesture commands, touch commands and voice commands;

[0072] S62, according to the user interaction operation information S user Specific instructions in the parse user's interaction needs and determine the operation type O type The operation types include browsing, zooming in, zooming out, rotating, switching, triggering interactive games, triggering knowledge quizzes, and triggering knowledge levels;

[0073] S63, for the determined operation type O type , the currently displayed intangible cultural heritage digital image content C i Conduct

[0074] The corresponding operation is performed to obtain the updated content C′ i ,Right now:

[0075] C′ i =δ(C i ,O type ),

[0076] Among them, δ represents the content response processing function, which can dynamically call sub-modules such as content transformation, animation control, and interface jump according to different operation types to achieve real-time update and feedback of AR content;

[0077] S64, updating the intangible cultural heritage digital image content C′ i The augmented reality rendering engine is used to overlay the display on the display interface of the user's terminal device to achieve dynamic content update and interactive feedback.

[0078] Optionally, step S7 specifically includes:

[0079] S71. During the user's interaction with the intangible cultural heritage digital image content, real-time collection of the user's learning data D learn and interactive behavior data D int The learning data includes the user's learning progress and knowledge mastery, and the interactive behavior data includes the user's interaction type, operation frequency and other information;

[0080] S72, collect the learning data D learn and interactive behavior data D int Analyze and process to generate learning feedback information F feedback , the formula is:

[0081] F feedback =η(D learn ,D int ),

[0082] Where η represents a data processing function that comprehensively analyzes user learning and interaction behaviors. The learning feedback information includes the user's learning performance, knowledge point mastery level, and interaction activity.

[0083] S73. Displaying learning results on the user terminal device based on the learning feedback information, including learning progress, knowledge point mastery, and interactive achievements;

[0084] S74. Based on the user's learning performance and interactive behavior, generate invitation information related to participating in intangible cultural heritage knowledge competitions and interactive question-answering education activities, and push it to the user's terminal device to promote the user's participation in further science popularization education activities.

[0085] Beneficial effects

[0086] This invention proposes an interactive system for intangible cultural heritage digital image science popularization based on augmented reality technology. Through the intangible cultural heritage digital resource acquisition and management module, it enables the efficient collection, editing, classification, and standardized management of various digital content related to intangible cultural heritage, including images, videos, 3D models, and animations. This significantly improves the integrity, scalability, and subsequent utilization efficiency of intangible cultural heritage resources. The augmented reality scene recognition module accurately captures information about specific objects, images, or geographic locations in the user's real-world environment, providing accurate contextual input for the content retrieval and recommendation module, ensuring that users receive personalized intangible cultural heritage digital content recommendations that are highly relevant to the actual scene. Through the augmented reality rendering and display module, relevant intangible cultural heritage image content can be intuitively overlaid on the user's terminal device interface in a virtual-realistic manner, achieving an immersive, multi-dimensional intangible cultural heritage science popularization experience. The multimodal interaction module supports multiple interaction methods, including gestures, touchscreen, voice, and facial expressions, significantly enhancing user engagement and interactive experience. The content dynamic response module updates and adjusts displayed content in real time based on the user's specific interactive operations, dynamically adapting content to user needs. The learning data collection and feedback module can comprehensively collect user learning process and behavior data, and combine intelligent analysis to generate learning feedback, personalized achievement display, and knowledge competition activity push, effectively promoting users' continuous learning and in-depth mastery of intangible cultural heritage knowledge. In summary, the present invention not only overcomes the problems of poor interactivity, insufficient immersion, and low personalization of traditional intangible cultural heritage popular science, but also greatly enhances the management, dissemination, and educational value of intangible cultural heritage digital content, promotes the innovative inheritance of intangible cultural heritage, expands social influence, and has significant application and promotion value. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0088] Figure 1 This is a schematic diagram of the structure and function of the intangible cultural heritage digital image popular science interactive system based on augmented reality technology proposed by the present invention;

[0089] Figure 2 This is a schematic diagram of the key technologies of the intangible cultural heritage digital image popular science interactive system in the present invention;

[0090] Figure 3 This is a user business flow diagram of the intangible cultural heritage digital image popular science interactive system based on augmented reality in the present invention. DETAILED DESCRIPTION

[0091] The present invention will now be described in further detail with reference to the accompanying drawings, which are simplified schematic diagrams that illustrate the basic structure of the present invention in a schematic manner.

[0092] refer to Figure 1-3 The intangible cultural heritage digital image popular science interactive system based on augmented reality technology includes:

[0093] The intangible cultural heritage digital resource collection and management module is used to collect data related to intangible cultural heritage, edit, process and store the data in a standardized manner, and establish an intangible cultural heritage digital resource library;

[0094] The augmented reality scene recognition module is used to identify the real scene through augmented reality technology on the user terminal device and obtain specific objects, images or geographic location information;

[0095] The content retrieval and recommendation module is used to retrieve and push intangible cultural heritage digital image content related to the scene or location from the intangible cultural heritage digital resource library based on the recognition results of the real scene or the user's geographical location information, thereby realizing personalized popular science knowledge recommendations;

[0096] The augmented reality rendering display module is used to overlay the retrieved intangible cultural heritage digital image content on the display interface of the user terminal device in an augmented reality manner, achieving a display effect that combines virtual and real;

[0097] The multimodal interaction module is used to provide a multimodal interaction interface on the user terminal device and obtain the user's interactive operation information;

[0098] The content dynamic response module is used to respond to user operations based on user interaction information and dynamically update the displayed intangible cultural heritage digital image content;

[0099] The learning data collection and feedback module is used to collect users' learning data and interactive behaviors in real time, generate learning feedback information, and display learning results on user terminal devices, or push intangible cultural heritage knowledge competitions and interactive question-answering educational activities.

[0100] In this embodiment, the modules are connected through the following methods:

[0101] S1. Collect and process images, videos, three-dimensional models, and animation data related to intangible cultural heritage, and perform standardized editing and storage on the data to build an intangible cultural heritage digital resource library;

[0102] S2. Using augmented reality scene recognition on the user terminal device, real-time images of the user's real environment or geographic location information are collected to obtain recognition results;

[0103] S3. Based on the recognition result or geographic location information, the content retrieval and recommendation module retrieves intangible cultural heritage digital image content related to the current scene or location in the intangible cultural heritage digital resource library, and performs relevance sorting and personalized recommendation;

[0104] S4, overlaying the retrieved intangible cultural heritage digital image content onto the display interface of the user terminal device through an augmented reality rendering display module, thereby realizing a digital content display that combines virtuality and reality;

[0105] S5, the multimodal interaction module obtains multimodal user interaction operation information including gesture recognition, touch screen operation, voice command and facial expression recognition, and integrates it into interaction instructions;

[0106] S6. The interactive instruction is parsed by the content dynamic response module according to the corresponding operation type, and the displayed intangible cultural heritage digital image content is browsed, enlarged, rotated, scaled, switched, or interactive games, knowledge quizzes, and knowledge challenge functions are triggered, and the display of the content is dynamically updated;

[0107] S7. Through the learning data collection and feedback module, the user's learning progress, interactive behavior and knowledge mastery are collected in real time, analyzed and generated learning feedback information, and learning results, personalized reports or invitations to intangible cultural heritage knowledge competitions and interactive educational activities are displayed on the user's terminal device.

[0108] In this embodiment, step S1 specifically includes:

[0109] S11, capturing images of intangible cultural heritage objects, performances, and scenes using digital camera equipment to obtain original image data G;

[0110] S12, using audio and video acquisition equipment to capture video of the intangible cultural heritage related performance and explanation process to obtain original video data V;

[0111] S13, using a 3D scanner or modeling software to perform 3D modeling on the intangible cultural heritage object to generate 3D model data M;

[0112] S14, animating the intangible cultural heritage related content using an animation production tool to obtain animation data A;

[0113] S15. Perform format conversion, editing, denoising, and enhancement processing on the acquired image data G, video data V, 3D model data M, and animation data A to obtain standardized digital content D. The formula is:

[0114] D=f(G,V,M,A),

[0115] Wherein, f represents the comprehensive function of digital content editing and processing;

[0116] S16. Classify, annotate and index the edited digital content D according to themes, types and sources, and establish an intangible cultural heritage digital resource library for subsequent retrieval and access.

[0117] In this embodiment, step S2 specifically includes:

[0118] S21, start the augmented reality recognition module on the user terminal device, and use the built-in camera to collect real-time scene image stream I;

[0119] S22. Apply an image recognition algorithm to extract features from the collected scene image stream I to obtain a scene feature vector F, namely:

[0120] F=φ(I),

[0121] Here, φ represents the image feature extraction function, which can extract key scene structure features and visual descriptions through convolutional neural networks (CNN), SIFT, or SURF mainstream visual feature extraction algorithms to achieve in-depth understanding and analysis of scene content;

[0122] S23, by comparing the scene feature vector F with the preset object and image feature database F db , determine whether there is a matching target, if so, determine the recognition result R of the specific object or image obj ,The comparison process can adopt vector distance nearest neighbor search or deep learning ,classification model;

[0123] S24. Obtain geographic location information L of the user terminal device, where the geographic location information includes latitude and longitude, wherein L is obtained by a global positioning system (GPS) or other positioning module, and L = (lat, lon), where lat represents latitude and lon represents longitude;

[0124] S25, the recognition result R obj Combined with geographic location information L, it serves as an input condition for subsequent retrieval and push of intangible cultural heritage digital image content, effectively improving the relevance and personalization of content to user scenarios.

[0125] In this embodiment, step S3 specifically includes:

[0126] S31, receiving the scene recognition result R output by the augmented reality recognition module obj and geographic location information L;

[0127] S32: Based on the scene recognition result and the geographical location information, the intangible cultural heritage digital image content C related to the scene or location information is searched in the intangible cultural heritage digital resource library. The search process is as follows:

[0128] C=Ψ(R obj ,L),

[0129] Among them, Ψ represents a resource retrieval function based on scene recognition and geographic location. It can comprehensively call keyword matching, content tag screening, and location similarity algorithms based on the user's current scene recognition results and geographic location to efficiently screen the intangible cultural heritage content in the database and recall content that is closely related to the actual scene;

[0130] S33. Sort the retrieved content C by relevance to obtain a sorted content set C. sort ,Right now:

[0131] C sort =Ω(C,R obj ,L),

[0132] Ω represents a content ranking function based on relevance and user preferences. It can perform multi-level weighted ranking based on multiple factors such as the relevance of content to the scene or location, user historical preferences, and content popularity, ensuring that the content pushed to users is the most relevant and prioritized.

[0133] S34, the sorted content set C sort Push to the user's terminal device and provide personalized scientific knowledge recommendations based on the user's current scenario or geographic location.

[0134] In this embodiment, step S4 specifically includes:

[0135] S41. Receive the sorted intangible cultural heritage digital image content set C on the user terminal device. sort ;

[0136] S42, using the augmented reality rendering display module to display the content set C sort Each item in C i Perform three-dimensional spatial positioning and determine the spatial coordinates P of the superimposed display i =(x i ,y i , z i ), where x i ,y i ,z i is the three-dimensional coordinate corresponding to the real scene;

[0137] S43, according to the scene image I and space coordinates P obtained in real time by the camera of the user terminal device i , using superposition rendering to transform the intangible cultural heritage digital image content C i It is superimposed on the real scene image I in the form of a virtual layer to generate an augmented reality display image A i ,Right now:

[0138] A i =Λ(I,C i ,P i ),

[0139] Here, Λ represents the augmented reality content overlay rendering function, which can achieve spatial projection, transparency adjustment, and dynamic interactive rendering of virtual content in real scenes through OpenGL and Unity3D three-dimensional rendering engines, ensuring that the display effect of virtual and real fusion is natural and realistic;

[0140] S44: Presenting augmented reality display image A on the display interface of the user terminal device i , allowing users to intuitively see the intangible cultural heritage digital image content that matches the spatial position of the real scene in the real environment, achieving a display effect that combines virtual and real.

[0141] In this embodiment, step S5 specifically includes:

[0142] S51. Initializing a multimodal interaction module on a user terminal device, wherein the multimodal interaction module supports three interaction modes: gesture recognition, touch screen operation, and voice command input;

[0143] S52, collecting user hand motion data G through gesture recognition sensor hand , and identify the gesture command S g ,Right now:

[0144] S g =α(G hand ),

[0145] Here, α represents the gesture recognition processing function, which can realize the classification and recognition of user gestures based on convolutional neural networks, time series analysis, or skeleton-based image processing algorithms;

[0146] S53, collecting user touch input data T through the touch screen touch , and identify and obtain the touch command S t , the formula is:

[0147] S t =β(T touch ),

[0148] Among them, β represents the touch input processing function, which can analyze multi-touch, sliding, clicking and other operations to achieve rich interface interactions;

[0149] S54, collecting user voice input data V through the microphone voice , and recognize the voice command S v , the formula is:

[0150] S v =γ(V voice ),

[0151] Here, γ represents the speech recognition processing function, which can call the ASR (automatic speech recognition) engine on the device or cloud to convert the user's natural voice commands into interactive commands that the system can recognize;

[0152] S55, the gesture command S g , touch command S t and voice commands S v Merge to form user interaction operation information S user ,Right now:

[0153] S user ={S g ,S t ,S v};

[0154] S56, the user interaction operation information S user Sent to the system interaction processing module as input conditions for subsequent intangible cultural heritage digital image content responses.

[0155] In this embodiment, step S6 specifically includes:

[0156] S61, receiving user interaction operation information S user , including gesture commands, touch commands and voice commands;

[0157] S62, according to the user interaction operation information S user Specific instructions in the parse user's interaction needs and determine the operation type O type The operation types include browsing, zooming in, zooming out, rotating, switching, triggering interactive games, triggering knowledge quizzes, and triggering knowledge levels;

[0158] S63, for the determined operation type O type , the currently displayed intangible cultural heritage digital image content C i Perform corresponding operations to obtain the updated content C′ i ,Right now:

[0159] C′ i =δ(C i ,O type ),

[0160] Among them, δ represents the content response processing function, which can dynamically call sub-modules such as content transformation, animation control, and interface jump according to different operation types to achieve real-time update and feedback of AR content;

[0161] S64, updating the intangible cultural heritage digital image content C′ i The augmented reality rendering engine is used to overlay the display on the display interface of the user's terminal device to achieve dynamic content update and interactive feedback.

[0162] In this embodiment, step S7 specifically includes:

[0163] S71. During the user's interaction with the intangible cultural heritage digital image content, real-time collection of the user's learning data D learn and interactive behavior data D int The learning data includes the user's learning progress and knowledge mastery, and the interactive behavior data includes the user's interaction type, operation frequency and other information;

[0164] S72, collect the learning data D learn and interactive behavior data D int Analyze and process to generate learning feedback information F feedback , the formula is:

[0165] F feedback =η(D learn ,D int ),

[0166] Where η represents a data processing function that comprehensively analyzes user learning and interaction behaviors. The learning feedback information includes the user's learning performance, knowledge point mastery level, and interaction activity.

[0167] S73. Displaying learning results on the user terminal device based on the learning feedback information, including learning progress, knowledge point mastery, and interactive achievements;

[0168] S74. Based on the user's learning performance and interactive behavior, generate invitation information related to participating in intangible cultural heritage knowledge competitions and interactive question-answering education activities, and push it to the user's terminal device to promote the user's participation in further science popularization education activities.

[0169] Example 1:

[0170] In order to verify the feasibility of the present invention in practice, the present invention was applied to the interactive activities of digital popular science of intangible cultural heritage in a historical and cultural block in Jiangsu Province. The local cultural department, in conjunction with intangible cultural heritage enterprises and universities, jointly organized the "Digital Intangible Cultural Heritage into the Block" experience month, aiming to enhance the popular science communication effect, interactivity and social influence of local intangible cultural heritage projects with the help of augmented reality technology. The block integrates a number of national and provincial intangible cultural heritage resources such as blue print printing and dyeing techniques, kites, paper-cutting, river and sea fishing customs. In the past, these intangible cultural heritage projects mostly used static exhibitions and explanations, lacking immersive and personalized experiences, and had a general response among young people. The dissemination of intangible cultural heritage knowledge faced problems such as low participation and lack of interest.

[0171] This embodiment uses the blue print dyeing technique as the core application scenario. During the event, the organizers set up augmented reality interactive experience points at multiple nodes in the block. Citizens and tourists can enter the intangible cultural heritage digital image popular science interactive system by simply using a smartphone or wearing AR glasses. In the experience area, users aim at exhibits, printing and dyeing tools, floral fabrics and other real objects, and the system automatically recognizes the scene and overlays three-dimensional animations, process demonstrations and interactive explanations on the screen. For example, when the user approaches the printing and dyeing process display area, the device immediately presents a complete process animation of blue print cloth from plate making, dyeing to drying, accompanied by dialect explanations and historical images, greatly enhancing the sense of immersion.

[0172] During the digital experience, users can use touchscreen, gestures, or voice to select different fabric patterns, zoom in on details, rotate them 360 degrees, and even try on traditional blue print clothing through AR, experiencing the folk costumes of the Jianghai region. The system also features interactive sessions such as a "Blue Print Quiz" and an "Intangible Cultural Heritage Challenge." For example, after watching an animation of the printing and dyeing process, the system automatically prompts the question: "Which plant is the traditional blue print dye primarily derived from?" Users can answer the question by voice or by clicking on an option. Correct answers receive instant feedback and are rewarded with points. Points can be redeemed for virtual crafts or a prize draw, further boosting learning motivation.

[0173] The system intelligently recommends relevant intangible cultural heritage content based on the user's location and interaction preferences. For example, in the kite craft experience area, the system prioritizes kite-making videos, 3D models, and expert presentations. After each interactive experience, the system automatically records the user's learning progress, answers, and interaction frequency, and analyzes the user's emotional state through facial expression recognition. If the user's interest is detected, the system automatically pushes more engaging interactions or invites them to participate in intangible cultural heritage knowledge competitions to enhance the personalized experience.

[0174] This event will be held from April 10, 2025 to May 10, 2025, with a period of one month. The specific application effects of the intangible cultural heritage digital image popular science interactive system are shown in Table 1.

[0175] Table 1: Statistical table of application effects of the intangible cultural heritage digital image popular science interactive system

[0176] index Specific data Comparison / Reference Total number of participants 22,105 people The traditional exhibitions during the same period attracted approximately 10,500 people. Proportion of teenagers and parents 41％ Traditional exhibitions about 20% Number of scene recognition tags 101,320 times / Number of pushed content 119,680 items / Average experience duration 16.5 minutes per person Traditional exhibition about 8.8 minutes Number of participants in interactive Q&A 34,860 times About 5,200 traditional exhibitions Correct answer rate 76％ Traditional method: about 48% Improvement rate of intangible cultural heritage knowledge 34％ / User satisfaction 92％ Traditional methods: about 69%

[0177] According to organizers' statistics, a total of 22,105 people participated in the intangible cultural heritage AR interactive experience during the event, of which 41% were young people and families. The system completed 101,320 real-world scene recognitions and delivered 119,680 pieces of intangible cultural heritage digital content. The average user experience lasted 16.5 minutes, an 88% increase compared to traditional exhibitions. A total of 34,860 participants participated in the interactive Q&A and knowledge challenge sessions, with an average accuracy rate of 76%. Following the event, a survey revealed a 34% increase in user knowledge of intangible cultural heritage, with 92% of respondents stating that the augmented reality interaction had enhanced their interest in and understanding of local intangible cultural heritage.

[0178] This embodiment takes the local intangible cultural heritage project "blue print dyeing skills" as the main line, and combines other representative intangible cultural heritage contents such as kites and paper-cuts. Through the multimodal interaction of augmented reality digital images, it significantly improves the fun, participation and immersion of intangible cultural heritage science popularization. The data show that the augmented reality system has significantly attracted more young people and parent-child families to participate, and the average experience time and interaction frequency are much higher than traditional exhibition methods. The improvement in the accuracy rate of interactive answering and the mastery of intangible cultural heritage knowledge is also very obvious, and user satisfaction has been greatly improved. Systematic data analysis and personalized content push effectively solve the problems of weak interactivity and limited dissemination effect of intangible cultural heritage science popularization activities, and provide practical technical support for the protection and dissemination of local intangible cultural heritage.

[0179] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. The intangible cultural heritage digital image popular science interactive system based on augmented reality technology is characterized by: include: The intangible cultural heritage digital resource collection and management module is used to collect data related to intangible cultural heritage, edit, process and store the data in a standardized manner, and establish an intangible cultural heritage digital resource library; The augmented reality scene recognition module is used to identify the real scene through augmented reality technology on the user terminal device and obtain specific objects, images or geographic location information; The content retrieval and recommendation module is used to retrieve and push intangible cultural heritage digital image content related to the scene or location from the intangible cultural heritage digital resource library based on the recognition results of the real scene or the user's geographical location information, thereby realizing personalized popular science knowledge recommendations; The augmented reality rendering display module is used to overlay the retrieved intangible cultural heritage digital image content on the display interface of the user terminal device in an augmented reality manner, achieving a display effect that combines virtual and real; The multimodal interaction module is used to provide a multimodal interaction interface on the user terminal device and obtain the user's interactive operation information; The content dynamic response module is used to respond to user operations based on user interaction information and dynamically update the displayed intangible cultural heritage digital image content; The learning data collection and feedback module is used to collect users' learning data and interactive behaviors in real time, generate learning feedback information, and display learning results on user terminal devices, or push intangible cultural heritage knowledge competitions and interactive question-answering educational activities.

2. The intangible cultural heritage digital image popular science interactive system based on augmented reality technology according to claim 1, characterized in that: The modules are implemented as follows: S1. Collect and process images, videos, three-dimensional models, and animation data related to intangible cultural heritage, and perform standardized editing and storage on the data to build an intangible cultural heritage digital resource library; S2. Using augmented reality scene recognition on the user terminal device, real-time images of the user's real environment or geographic location information are collected to obtain recognition results; S3. Based on the recognition result or geographic location information, the content retrieval and recommendation module retrieves intangible cultural heritage digital image content related to the current scene or location in the intangible cultural heritage digital resource library, and performs relevance sorting and personalized recommendation; S4, overlaying the retrieved intangible cultural heritage digital image content onto the display interface of the user terminal device through an augmented reality rendering display module, thereby realizing a digital content display that combines virtuality and reality; S5, the multimodal interaction module obtains multimodal user interaction operation information including gesture recognition, touch screen operation, voice command and facial expression recognition, and integrates it into interaction instructions; S6. The interactive instruction is parsed by the content dynamic response module according to the corresponding operation type, and the displayed intangible cultural heritage digital image content is browsed, enlarged, rotated, scaled, switched, or interactive games, knowledge quizzes, and knowledge challenge functions are triggered, and the display of the content is dynamically updated; S7. Through the learning data collection and feedback module, the user's learning progress, interactive behavior and knowledge mastery are collected in real time, analyzed and generated learning feedback information, and learning results, personalized reports or invitations to intangible cultural heritage knowledge competitions and interactive educational activities are displayed on the user's terminal device.

3. The intangible cultural heritage digital image popular science interactive system based on augmented reality technology according to claim 2, characterized in that: The step S1 specifically includes: S11. Capturing images of objects, performances, and scenes of intangible cultural heritage using digital camera equipment to obtain original image data; S12. Capture video of the intangible cultural heritage related performances and explanations using audio and video acquisition equipment to obtain original video data; S13. Use a 3D scanner or modeling software to perform 3D modeling on the intangible cultural heritage object to generate 3D model data; S14, animating the intangible cultural heritage related content using an animation production tool to obtain animation data; S15, performing format conversion, editing, denoising, and enhancement processing on the collected image data, video data, 3D model data, and animation data to obtain standardized digital content; S16. Classify, label and index the edited digital content according to themes, types and sources to establish an intangible cultural heritage digital resource library.

4. The intangible cultural heritage digital image popular science interactive system based on augmented reality technology according to claim 2, characterized in that: The step S2 specifically includes: S21. Start the augmented reality scene recognition module on the user terminal device and use the built-in camera to collect real-time scene image stream; S22, applying an image recognition algorithm to perform feature extraction on the collected scene image stream to obtain a scene feature vector; S23, by comparing the scene feature vector with a preset object and image feature database, determining whether there is a matching target, and if so, determining the recognition result of the specific object or image; S24. Obtaining geographic location information of the user terminal device, where the geographic location information includes latitude and longitude, and is obtained by a global positioning system (GPS) or other positioning module; S25. Combining the recognition result with the geographic location information as input conditions for subsequent retrieval and push of intangible cultural heritage digital image content.

5. The intangible cultural heritage digital image popular science interactive system based on augmented reality technology according to claim 2, characterized in that: The step S3 specifically includes: S31, receiving the scene recognition result and geographic location information output by the augmented reality scene recognition module; S32, searching the intangible cultural heritage digital image content related to the scene or location information in the intangible cultural heritage digital resource library based on the scene recognition result and the geographical location information; S33, sorting the retrieved intangible cultural heritage digital image contents by relevance to obtain a sorted content set; S34: Push the sorted content collection to the user's terminal device, and implement personalized popular science knowledge recommendations based on the user's current scenario or geographic location.

6. The intangible cultural heritage digital image popular science interactive system based on augmented reality technology according to claim 2, characterized in that: The step S4 specifically includes: S41, receiving the sorted intangible cultural heritage digital image content set on the user terminal device; S42, performing three-dimensional spatial positioning of each content item in the content set through the augmented reality rendering display module to determine the spatial coordinates of the superimposed display; S43, based on the scene image and spatial coordinates acquired in real time by the camera of the user terminal device, superimpose the intangible cultural heritage digital image content on the real scene image in the form of a virtual layer using an overlay rendering method to generate an augmented reality display image; S44. Presenting an augmented reality display image on the display interface of the user terminal device, so that the user can intuitively see the intangible cultural heritage digital image content that matches the spatial position of the real scene in the real environment picture.

7. The intangible cultural heritage digital image popular science interactive system based on augmented reality technology according to claim 2, characterized in that: The step S5 specifically includes: S51. Initializing a multimodal interaction module on a user terminal device, wherein the multimodal interaction module supports three interaction modes: gesture recognition, touch screen operation, and voice command input; S52, collecting user hand motion data through a gesture recognition sensor and identifying and obtaining gesture instructions; S53, collecting user touch input data through the touch screen and identifying it to obtain touch instructions; S54, collecting user voice input data through a microphone and recognizing it to obtain a voice command; S55, combining the gesture command, the touch command, and the voice command to form user interaction operation information; S56. Send the user interaction operation information to the system interaction processing module as an input condition for the subsequent intangible cultural heritage digital image content response.

8. The intangible cultural heritage digital image popular science interactive system based on augmented reality technology according to claim 2, characterized in that: The step S6 specifically includes: S61, receiving user interaction operation information, including gesture commands, touch commands, and voice commands; S62: Analyze the user's interaction needs based on the specific instructions in the user interaction operation information and determine the operation type, wherein the operation type includes browsing, zooming in, zooming out, rotating, switching, triggering an interactive game, triggering a knowledge quiz, and triggering a knowledge challenge; S63: performing corresponding operations on the currently displayed intangible cultural heritage digital image content according to the determined operation type to obtain updated content; S64. The updated intangible cultural heritage digital image content is re-superimposed and displayed on the display interface of the user terminal device through the augmented reality rendering engine to achieve dynamic update of the content and interactive feedback.

9. The intangible cultural heritage digital image popular science interactive system based on augmented reality technology according to claim 2, characterized in that: The step S7 specifically includes: S71. During the user's interaction with the intangible cultural heritage digital image content, real-time collection of the user's learning data and interaction behavior data, wherein the learning data includes the user's learning progress and knowledge mastery, and the interaction behavior data includes the user's interaction type and operation frequency; S72: Analyze and process the collected learning data and interactive behavior data to generate learning feedback information, which includes the user's learning performance, knowledge point mastery level, and interactive activity; S73. Displaying learning results on the user terminal device based on the learning feedback information, including learning progress, knowledge point mastery, and interactive achievements; S74. Based on the user's learning performance and interactive behavior, generate invitation information related to participating in intangible cultural heritage knowledge competitions and interactive question-answering education activities, and push it to the user's terminal device to promote the user's participation in further science popularization education activities.

Images