Immersive art display system and method based on superposition of AR projection and entity traditional Chinese painting

Abstract

The invention discloses an AR projection and entity traditional Chinese painting superposed immersive art display system and method, and the system and method achieve the seamless superposition of an entity traditional Chinese painting and a virtual picture through the combination of an AR technology and a digital projection technology, and enhance the immersive experience and interactivity of audiences. The system comprises an entity traditional Chinese painting display module, an AR projection module, an interaction module and a data processing and storage module. The method comprises the steps that the AR recognition device collects feature information of an entity traditional Chinese painting, the projection control unit generates virtual projection content and projects the virtual projection content to the surface of the painting, and audiences interact with the system through gesture, voice or touch operation to adjust the projection content in real time. Example verification shows that the system can superpose the effects of dynamic cloud and mist, flying birds and the like, audiences can amplify details and inquire creation backgrounds, and the enjoyment and cultural connotation of appreciation are improved. The problem that traditional Chinese painting display lacks interactivity is solved, an innovative scheme is provided for fusion of traditional art and modern science and technology, and the method is suitable for museums, exhibition halls and other scenes.

Description

Technical Field

[0001] This invention relates to the intersection of augmented reality (AR) technology, digital projection technology, and traditional art display, specifically to an immersive art display system and method that overlays AR projection with a physical Chinese painting. Background Technology

[0002] Traditional Chinese painting, as an important carrier of Chinese traditional culture, possesses profound artistic value and cultural connotations. However, the display methods of traditional Chinese paintings are mostly static hanging or mounting, allowing viewers to only perceive the content visually, lacking interactivity and immersion. With the development of AR and digital projection technologies, how to combine modern technology with traditional art to create a brand-new form of art display has become a research hotspot in the field of cultural and technological integration.

[0003] In existing technologies, AR technology has been widely used in museums, exhibition halls, and other settings, but it mainly focuses on displaying virtual exhibits or virtual scenes, lacking deep integration with physical artworks. While digital projection technology can achieve dynamic image display, it is difficult to seamlessly overlay with physical artworks and lacks interactivity. Therefore, there is an urgent need for an immersive art display system and method that can overlay AR projections with physical Chinese paintings to enhance the viewing experience and artistic expression of traditional Chinese paintings. Summary of the Invention

[0004] The purpose of this invention is to provide an immersive art display system and method that overlays AR projection with physical Chinese paintings. By combining AR technology and digital projection technology, it achieves precise overlay of physical Chinese paintings with virtual images, enhances the audience's immersive experience and interactivity, and improves the artistic expression and cultural dissemination effect of traditional Chinese paintings.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] An immersive art display system that overlays AR projections with physical traditional Chinese paintings includes:

[0007] The physical Chinese painting display module is used to display physical Chinese painting works, and the surface of the physical Chinese painting has identifiable feature information;

[0008] The AR projection module includes a projector, an AR recognition device, and a projection control unit. The AR recognition device is used to collect feature information of the physical Chinese painting. The projection control unit generates virtual projection content based on the feature information and projects the virtual projection content onto the surface of the physical Chinese painting through the projector, thereby achieving the superposition of the virtual image and the physical Chinese painting.

[0009] An interactive module is used to receive interactive instructions from the audience and transmit the interactive instructions to the projection control unit to adjust the virtual projection content or trigger dynamic effects.

[0010] The data processing and storage module is used to store the digital information of the physical Chinese painting and the virtual projection content, and to generate and output the adjusted projection content in real time according to the interactive instructions.

[0011] Furthermore, the feature information of the physical Chinese painting includes at least one of the following:

[0012] Specific textures, markings, or QR codes on the surface of physical Chinese paintings;

[0013] Image features or spatial location information of physical Chinese paintings.

[0014] Furthermore, the AR recognition device is at least one of the following:

[0015] A camera is used to capture image information of physical Chinese paintings;

[0016] Depth sensor is used to collect spatial location information of the physical Chinese painting;

[0017] A QR code scanner is used to identify the QR code information on the surface of a physical Chinese painting.

[0018] Furthermore, the interaction module is at least one of the following:

[0019] Gesture recognition devices are used to recognize the hand gestures of the audience;

[0020] Voice recognition equipment is used to recognize the voice commands of the audience;

[0021] Touchscreen devices are used to receive touch input from viewers.

[0022] Furthermore, the virtual projection content includes at least one of the following:

[0023] Dynamic effects, including animations of clouds, flowing water, and flying birds;

[0024] Interactive scripts are used to respond to audience interaction commands and trigger dynamic effects;

[0025] Art analysis information, including the creative background, technical analysis, or explanation of cultural connotations.

[0026] An immersive art display method that overlays AR projection with physical traditional Chinese paintings includes the following steps:

[0027] Physical Chinese painting recognition steps: Collect feature information of the physical Chinese painting through AR recognition device, and transmit the feature information to projection control unit;

[0028] Projection content generation steps: The projection control unit retrieves the corresponding virtual projection content from the data processing and storage module according to the feature information, and generates an overlay image by combining it with the digital information of the physical Chinese painting;

[0029] Projection overlay step: The virtual projection content is projected onto the surface of the physical Chinese painting using a projector, so as to achieve precise overlay of the virtual image and the physical Chinese painting;

[0030] Audience interaction steps: The interaction module receives the audience's interaction instructions and transmits the interaction instructions to the projection control unit; the projection control unit adjusts the virtual projection content or triggers dynamic effects according to the interaction instructions.

[0031] Furthermore, in the physical Chinese painting recognition step, the feature information is collected in the following ways:

[0032] Image information of physical Chinese paintings is captured using a camera;

[0033] Spatial location information of the physical Chinese painting is collected using a depth sensor;

[0034] The QR code information on the surface of the physical Chinese painting is identified using a QR code scanner.

[0035] Furthermore, in the projection content generation step, the virtual projection content is generated according to at least one of the following methods:

[0036] Match the corresponding dynamic effects to the artistic style or theme of the physical Chinese painting;

[0037] Based on the audience's interactive instructions, the preset interactive script is invoked;

[0038] Based on the audience's query request, output art analysis information.

[0039] Furthermore, in the audience interaction step, the interaction instructions include at least one of the following:

[0040] Gesture commands, including zooming in, zooming out, and rotating the screen;

[0041] Voice commands, including querying art analysis information or switching projection content;

[0042] Touch commands, including clicking on the screen to trigger dynamic effects.

[0043] Furthermore, the method also includes the following steps:

[0044] Data storage and update steps: Store the digital information of the physical Chinese painting and the virtual projection content in the data processing and storage module, and update the virtual projection content as needed.

[0045] The immersive art display system and method of superimposing AR projection with physical traditional Chinese paintings of the present invention has the following beneficial effects:

[0046] Immersive experience: By overlaying AR projection with physical Chinese paintings, viewers can simultaneously perceive the texture of the physical artwork and the dynamic effects of the virtual image, enhancing the immersive and engaging experience.

[0047] Enhanced interactivity: Viewers can interact with the display system through gestures, voice, or touch operations, triggering dynamic changes in the virtual images or obtaining more artistic information, thus enhancing the enjoyment and participation of the viewing experience.

[0048] Enhanced cultural dissemination: By showcasing the creative background and artistic analysis of traditional Chinese paintings through virtual projection, viewers can gain a deeper understanding of the cultural connotations and artistic value of these paintings, thereby enhancing the effectiveness of cultural dissemination.

[0049] Protection and Inheritance: Recording and displaying physical Chinese paintings through digital means reduces direct contact and damage to the original works, which is conducive to the protection and inheritance of traditional Chinese paintings. Attached Figure Description

[0050] Figure 1 This is a schematic diagram of the system architecture of the immersive art display system that overlays AR projection with a physical Chinese painting in this invention.

[0051] Figure 2 This is a schematic diagram illustrating the implementation process of the immersive art display method that overlays AR projection with a physical traditional Chinese painting in this invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0053] I. System Composition

[0054] This invention provides an immersive art display system that overlays AR projection with a physical traditional Chinese painting, comprising the following modules:

[0055] Physical Chinese painting display module: used to display physical Chinese painting works, the surface of which has identifiable feature information, including specific textures, marks or QR codes;

[0056] AR projection module: includes a projector, AR recognition device and projection control unit;

[0057] AR recognition devices are used to collect feature information of physical Chinese paintings, including cameras (to collect image information), depth sensors (to collect spatial location information), or QR code scanners (to recognize QR code information).

[0058] Projection control unit: Generates virtual projection content based on the feature information, and projects the virtual projection content onto the surface of the physical Chinese painting through the projector, thereby achieving the superposition of the virtual image and the physical Chinese painting;

[0059] Interaction module: Used to receive interactive instructions from the audience, including gesture recognition devices (recognizing gesture actions), voice recognition devices (recognizing voice instructions), or touch devices (receiving touch operations), and transmit the interactive instructions to the projection control unit to adjust the virtual projection content or trigger dynamic effects;

[0060] Data processing and storage module: used to store the digital information of the physical Chinese painting (high-definition image, artistic analysis, creation background) and the virtual projection content (dynamic effects, interactive script), and to generate and output the adjusted projection content in real time according to the interactive instructions.

[0061] II. Method and Flow

[0062] This invention also provides an immersive art display method that overlays AR projection with a physical traditional Chinese painting, including the following steps:

[0063] Physical Chinese painting recognition steps: Collect feature information of the physical Chinese painting through AR recognition device, and transmit the feature information to projection control unit;

[0064] Projection content generation steps: The projection control unit retrieves the corresponding virtual projection content from the data processing and storage module according to the feature information, and generates an overlay image by combining it with the digital information of the physical Chinese painting;

[0065] Projection overlay step: The virtual projection content is projected onto the surface of the physical Chinese painting using a projector, so as to achieve precise overlay of the virtual image and the physical Chinese painting;

[0066] Audience interaction steps: The interaction module receives the audience's interaction instructions and transmits the interaction instructions to the projection control unit; the projection control unit adjusts the virtual projection content or triggers dynamic effects according to the interaction instructions.

[0067] III. Implementation Methods

[0068] 1. System Setup

[0069] The immersive art display system of this invention requires the integration and construction of hardware and software in scenarios such as exhibition halls or museums. The specific steps are as follows:

[0070] (1) Setting up a physical Chinese painting exhibition area

[0071] Exhibit preparation: Select representative physical Chinese paintings and ensure that the surface of the paintings has identifiable features;

[0072] Feature type:

[0073] Specific textures: The distinctive brushstrokes, seals, or framing textures in a painting;

[0074] Marking: Add invisible markings (infrared markings or fluorescent markings) to the edges or blank areas of the artwork;

[0075] QR code: Set up QR code labels in the painting mounting area or display case. The QR code information may include the painting ID and the digital information storage address.

[0076] Exhibition environment:

[0077] Hang physical Chinese paintings on the walls of the exhibition hall or place them in display cases, ensuring that the surface of the paintings is unobstructed, so that AR recognition devices can collect feature information.

[0078] Control the lighting conditions in the exhibition hall to avoid direct strong light that could cause failure to collect feature information (QR code reflection, blurred texture).

[0079] (2) Installation and debugging of AR projection module

[0080] Hardware components:

[0081] Projector: Choose a high-brightness, high-resolution projector to ensure the clarity of the virtual images in the exhibition hall environment;

[0082] AR recognition devices:

[0083] Camera: Used to capture image information of the painting, it must have high resolution and wide-angle shooting capabilities;

[0084] Depth sensor: Used to acquire spatial position information (distance, angle) of the painting to ensure accurate superposition of the virtual image and the physical painting;

[0085] QR code scanner: Used to quickly identify QR code information on the surface of a painting;

[0086] Projection control unit: Uses a high-performance computer or embedded device to run projection control software;

[0087] Installation and debugging:

[0088] Install the projector on the top or side of the exhibition hall, and adjust the projection angle and focus to ensure that the virtual image covers the surface of the physical Chinese painting.

[0089] Fix the AR recognition device to the top of the exhibition hall or around the display case to ensure that its field of vision covers the physical Chinese painting display area;

[0090] Spatial calibration of the projector and AR recognition device is performed using calibration tools (including a checkerboard calibration board) to eliminate projection distortion and recognition errors.

[0091] (3) Interactive module configuration

[0092] Hardware components:

[0093] Gesture recognition camera: Uses a depth camera (Kinect) or a structured light camera to capture audience gestures in real time;

[0094] Voice recognition microphone: Employs an array microphone to improve the accuracy of voice command recognition;

[0095] Touchscreens: Touchscreens are installed in the exhibition hall, allowing visitors to interact with the system via touch.

[0096] Connection and debugging:

[0097] Connect the interactive module to the projection control unit via wired or wireless means;

[0098] Configure the parameters of the interactive module (including gesture recognition sensitivity and voice command keywords) to ensure smooth operation for the audience.

[0099] (4) Data processing and storage module construction

[0100] Hardware components:

[0101] Use servers or cloud storage platforms to store digital information and virtual projection content of physical Chinese paintings;

[0102] Software development:

[0103] Develop a database management system to store digital information such as high-resolution images of paintings, creation backgrounds, and technique analyses;

[0104] Develop software to generate projection content dynamically based on the characteristics of the artwork (dynamic effects, interactive scripts);

[0105] Develop control software to receive data from AR recognition devices and interactive modules, and adjust the projected content in real time;

[0106] Data updates and maintenance:

[0107] Regularly update virtual projection content (add dynamic effects, optimize interactive scripts);

[0108] Back up the digital information of the physical Chinese painting to prevent data loss.

[0109] 2. Operating Procedures

[0110] The operation process of this invention includes the following steps to ensure that viewers can obtain an immersive experience through the system:

[0111] (1) Identification of physical Chinese paintings

[0112] Once visitors enter the exhibition hall, the AR recognition device automatically activates to collect feature information of the physical Chinese paintings;

[0113] Image acquisition: The camera captures images of the painting, and features such as texture and markings are extracted;

[0114] Spatial positioning: Depth sensors acquire spatial location information of the artwork;

[0115] QR code recognition: A QR code scanner reads the QR code information on the surface of the painting;

[0116] The feature information is transmitted to the projection control unit for parsing and matching.

[0117] (2) Projection content generation and overlay

[0118] The projection control unit retrieves the corresponding virtual projection content from the data processing and storage module based on the feature information;

[0119] Dynamic effects generation: Generate dynamic elements (such as clouds, birds, and flowing water) based on the theme of the painting (such as landscapes, flowers and birds);

[0120] Interactive script loading: Loads preset interactive scripts, defining the correspondence between audience actions and projected content;

[0121] The projector projects virtual images onto the surface of a physical Chinese painting, achieving a precise overlay of the virtual and the physical.

[0122] (3) Audience interaction

[0123] Viewers can interact with the system in the following ways:

[0124] Gesture control: The camera captures hand gestures, such as zooming in, zooming out, and rotating the image;

[0125] Voice commands: Input commands via the voice recognition microphone, such as "Query the background of the creation" or "Switch the dynamic effect";

[0126] Touch operation: Tap the screen to trigger specific dynamic effects or obtain art analysis information.

[0127] The interactive module transmits audience commands to the projection control unit.

[0128] (4) Real-time adjustment of projected content

[0129] The projection control unit adjusts the virtual projection content in real time according to the audience's instructions.

[0130] Dynamic effects adjustment: such as changing the flow speed of clouds and the flight path of birds;

[0131] Information display: such as pop-up text or image information about the artwork's creation background and technique analysis;

[0132] Interactive feedback: such as prompting the audience that the operation was successful through sound effects or light effects.

[0133] 3. Example Verification

[0134] Taking a traditional landscape painting as an example, the actual effect of the present invention is verified:

[0135] (1) System Configuration

[0136] A traditional landscape painting is hung in the exhibition hall, with invisible markings and QR codes added to its surface;

[0137] Install a projector, AR recognition equipment (camera and depth sensor), gesture recognition camera and touch screen;

[0138] Build a data processing and storage module to store digital information of the painting and virtual projection content (such as clouds, birds, and flowing water animation).

[0139] (2) Operation process demonstration

[0140] Identification phase:

[0141] After visitors enter the exhibition hall, cameras and depth sensors automatically capture images and spatial information of the paintings, and QR code scanners read the QR code information.

[0142] The projection control unit parses the feature information and loads the corresponding virtual projection content;

[0143] Projection overlay stage:

[0144] The projector projects virtual images onto the surface of the painting, allowing viewers to see the superimposed effect of real landscape outlines and dynamic clouds, mist, and flying birds;

[0145] Interactive phase:

[0146] Viewers can use gestures to zoom in on details in the painting and examine the painter's brushstrokes and ink variations;

[0147] Viewers can use voice commands to "inquire about the background of the work," and the system will display text information introducing the creation date of the painting, the artist's life and artistic style.

[0148] Viewers can tap the "waterfall" area in the painting on a touchscreen to trigger dynamic waterfall effects and water flow sound effects.

[0149] (3) Effect evaluation

[0150] Immersive experience: Viewers reported that the superposition of virtual images and physical paintings was natural, and the dynamic effects enhanced the vividness of the images;

[0151] Interactivity: Gestures, voice and touch operations are highly responsive, allowing viewers to easily access artistic information;

[0152] Cultural dissemination: The presentation of art analysis information helps viewers to understand the cultural connotations of the paintings more deeply, enhancing the enjoyment and educational significance of the viewing experience.

[0153] The present invention and its embodiments have been described above. This description is not restrictive, and the accompanying drawings are only one embodiment of the present invention; the actual structure is not limited thereto. In conclusion, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the invention, such designs should fall within the protection scope of the present invention.

Claims

1. An immersive art display system that overlays AR projection with a physical traditional Chinese painting, characterized in that: include: The physical Chinese painting display module is used to display physical Chinese painting works, and the surface of the physical Chinese painting has identifiable feature information; The AR projection module includes a projector, an AR recognition device, and a projection control unit. The AR recognition device is used to collect feature information of the physical Chinese painting. The projection control unit generates virtual projection content based on the feature information and projects the virtual projection content onto the surface of the physical Chinese painting through the projector, thereby achieving the superposition of the virtual image and the physical Chinese painting. An interactive module is used to receive interactive instructions from the audience and transmit the interactive instructions to the projection control unit to adjust the virtual projection content or trigger dynamic effects. The data processing and storage module is used to store the digital information of the physical Chinese painting and the virtual projection content, and to generate and output the adjusted projection content in real time according to the interactive instructions.

2. The immersive art display system that overlays AR projection with a physical traditional Chinese painting according to claim 1, characterized in that: The feature information of the physical Chinese painting includes at least one of the following: Specific textures, markings, or QR codes on the surface of physical Chinese paintings; Image features or spatial location information of physical Chinese paintings.

3. The immersive art display system that overlays AR projection with a physical traditional Chinese painting according to claim 1, characterized in that: The AR recognition device is at least one of the following: A camera is used to capture image information of physical Chinese paintings; Depth sensor is used to collect spatial location information of the physical Chinese painting; A QR code scanner is used to identify the QR code information on the surface of a physical Chinese painting.

4. The immersive art display system that overlays AR projection with a physical traditional Chinese painting according to claim 1, characterized in that: The interaction module is at least one of the following: Gesture recognition devices are used to recognize the hand gestures of the audience; Voice recognition equipment is used to recognize the voice commands of the audience; Touchscreen devices are used to receive touch input from viewers.

5. The immersive art display system that overlays AR projection with a physical traditional Chinese painting according to claim 1, characterized in that: The virtual projection content includes at least one of the following: Dynamic effects, including animations of clouds, flowing water, and flying birds; Interactive scripts are used to respond to audience interaction commands and trigger dynamic effects; Art analysis information, including the creative background, technical analysis, or explanation of cultural connotations.

6. An immersive art display method that overlays AR projection with a physical traditional Chinese painting, characterized in that... Includes the following steps: Physical Chinese painting recognition steps: Collect feature information of the physical Chinese painting through AR recognition device, and transmit the feature information to projection control unit; Projection content generation steps: The projection control unit retrieves the corresponding virtual projection content from the data processing and storage module according to the feature information, and generates an overlay image by combining it with the digital information of the physical Chinese painting; Projection overlay step: The virtual projection content is projected onto the surface of the physical Chinese painting using a projector, so as to achieve precise overlay of the virtual image and the physical Chinese painting; Audience interaction steps: The interaction module receives the audience's interaction instructions and transmits the interaction instructions to the projection control unit; the projection control unit adjusts the virtual projection content or triggers dynamic effects according to the interaction instructions.

7. The immersive art display method of superimposing AR projection with a physical traditional Chinese painting according to claim 6, characterized in that: In the physical Chinese painting recognition step, the feature information is collected in the following ways: Image information of physical Chinese paintings is captured using a camera; Spatial location information of the physical Chinese painting is collected using a depth sensor; The QR code information on the surface of the physical Chinese painting is identified using a QR code scanner.

8. The immersive art display method of superimposing AR projection with a physical traditional Chinese painting according to claim 6, characterized in that: In the projection content generation step, the virtual projection content is generated according to at least one of the following methods: Match the corresponding dynamic effects to the artistic style or theme of the physical Chinese painting; Based on the audience's interactive instructions, the preset interactive script is invoked; Based on the audience's query request, output art analysis information.

9. The immersive art display method of superimposing AR projection with a physical traditional Chinese painting according to claim 6, characterized in that: In the audience interaction step, the interaction instructions include at least one of the following: Gesture commands, including zooming in, zooming out, and rotating the screen; Voice commands, including querying art analysis information or switching projection content; Touch commands, including clicking on the screen to trigger dynamic effects.

10. The immersive art display method of superimposing AR projection with a physical traditional Chinese painting according to claim 6, characterized in that: The method further includes the following steps: Data storage and update steps: Store the digital information of the physical Chinese painting and the virtual projection content in the data processing and storage module, and update the virtual projection content as needed.

Images