Interaction system for generating culture and art based on vibration drum beating

Through the interactive system combining high-sensitivity sensors and Stable Diffusion model, the problems of limitations of application scenarios and experience distance in traditional lion-drawing cultural experience are solved, and rapid response and personalized cultural pattern generation are achieved, which improves user experience and culture Transmission effect.

CN119065772APending Publication Date: 2024-12-03SHANGHAI FENGYUZHU CULTURE TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411196295.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-29
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The existing technology has limitations in the traditional lion-drawing cultural experience and the distance problem of experience. Real-time response and data processing speed are insufficient, making it difficult to effectively integrate traditional Chinese cultural elements with modern interactive design, and the user experience is not smooth enough.

Method used

Using a high-sensitivity pressure sensor array to capture drum beats, combining Stable Diffusion model and high-speed serial communication technology to generate a personalized lion-draw pattern and project through high-resolution projectors to provide an immersive experience while integrating QR code Generate modules for user sharing.

Benefits of technology

Improve the response speed and accuracy of drum beat interaction, enhance the stability and efficiency of data transmission, realize the rapid generation of personalized experiences with Chinese characteristics, shorten the distance between users and culture, and promote the cultural Disseminate and protect.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119065772A_ABST
    Figure CN119065772A_ABST
Patent Text Reader

Abstract

The invention provides an interactive system for generating culture and art based on vibration drum beating, and relates to the field of traditional culture and art. According to the interactive system for generating the traditional culture and art based on the vibration drum striking, a software technical scheme and a hardware technical scheme work cooperatively, and the software technical scheme and the hardware technical scheme comprise drum head striking and characteristic lion striking elements. The drum beating interaction speed and accuracy are improved through a high-precision sensor and a data processing algorithm, the communication technology is optimized, the data transmission stability and efficiency are enhanced, and high-speed information transmission in a complex environment is guaranteed. The AIGC technology and the Stable Diffusion model are adopted, the image generation calculation requirement is lowered, and Chinese characteristic patterns such as striking are rapidly generated. The user-friendly interactive design shortens the experience distance, and is convenient for users to experience and learn culture in non-traditional occasions. Cultural elements and interactive experience are innovatively combined, a two-dimensional code sharing function enables a user to easily store and spread a lion-striking pattern, and culture popularization is facilitated.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of traditional culture and art, and in particular to an interactive system for generating culture and art based on vibration drumming. Background Art

[0002] By beating drums or other vibrating devices, specific vibrations and sounds are generated. These sounds can be used to express cultural art forms such as traditional music, dance or drama. For example, in traditional drum performances, the vibration and sound of the drums are key artistic elements. Some technologies can transform sound or vibration into visual artistic effects, such as using the vibration patterns of sound waves to form beautiful patterns on sand, liquid or other media. These patterns can reflect the symbols or patterns of traditional culture, thereby creating unique artistic effects, and using modern technologies (such as sensors, data analysis and computer-generated images) to enhance the expressiveness of traditional cultural arts. For example, using vibration sensors to capture the dynamic changes of drum sounds, and then using computers to generate corresponding visual effects to create new artistic experiences. In general, this method attempts to explore and express the deep connotations of traditional cultural arts through vibrations and sound waves, integrating modern technology and traditional art to produce new art forms and experiences.

[0003] Existing interactive devices integrate advanced sensor technology, data processing, communication technology, deep learning image generation algorithms, and QR code generation technology. The combination of these technologies provides users with an innovative interactive experience. However, existing technologies still have limitations in real-time response, data processing speed, naturalness of user interaction, and accurate expression of cultural elements. For example, the accuracy and stability of sensors, the rate and anti-interference ability of serial communication, server processing capabilities, and the high demand for computing resources of image generation algorithms are all possible technical bottlenecks. In addition, how to effectively integrate traditional Chinese cultural elements such as the lion dance pattern into modern interactive design while ensuring the smoothness of the user experience and cultural suitability is also a challenge that needs to be overcome in the design. Summary of the invention

[0004] 1. Technical issues to be resolved

[0005] In view of the shortcomings of the prior art, the present invention provides an interactive system for generating cultural arts based on vibration drumming, which solves the application scenario limitations and experience distance problems existing in the traditional lion awakening cultural experience method, and provides an immersive digital experience platform through the innovative combination of interactive hardware and software.

[0006] (II) Technical solution

[0007] To achieve the above purpose, the present invention is implemented through the following technical solutions: an interactive system based on vibration drumming to generate cultural arts, including software technical solutions and hardware technical solutions working together, the software technical solutions and hardware technical solutions include striking drum surface and characteristic lion awakening elements, the striking drum surface is connected to a server receiving rate through serial port communication, the server receiving rate is connected to a judgment of striking frequency and a judgment of the number of striking times, the judging striking frequency is connected to an interactive module through a rhythm-formed specific mask, the judging number of striking times transmits the number of striking times to Huawei for text encoding to the corresponding prompt words and is connected to the interactive module;

[0008] The characteristic lion awakening element is connected to a Stable Diffusion model, the server receiving rate is connected to the Stable Diffusion model, and the Stable Diffusion model is connected to the interactive module;

[0009] The interactive module includes input image encoding and post-processing, the interactive module generates a custom generated image with lion awakening elements, the custom generated image with lion awakening elements is connected to an overlay interface design module, the overlay interface design template is connected to a generated QR code, and the generated QR code is connected to an object projected on the counter-projection.

[0010] Preferably, the input image encoding is connected with noise addition, the noise addition is connected with text encoding, the text encoding is connected with a sampling process, the sampling process is connected with a UNet network structure, the UNet network structure is connected with latent space decoding, the post-processing is connected with control and fine-tuning, and the control and fine-tuning are connected with iterative optimization.

[0011] Preferably, the hardware technical solution includes:

[0012] Drumming device: A highly sensitive pressure sensor array is installed under the drum surface to capture the force and position of the experiencer's drumming. The sensors are arranged in a grid to ensure full coverage of all areas of the drum surface.

[0013] Sensor and data processing unit: The data collected by the sensor is converted into digital signals through an analog-to-digital converter (ADC) and transmitted to the data processing unit through a high-speed serial communication interface;

[0014] Server host: The server host is equipped with high-performance CPU and GPU to run the Stable Diffusion image generation model, and has powerful data processing and storage capabilities;

[0015] Projector and drum surface decoration: The high-resolution projector projects the interactive interface and generated cultural patterns and lion dance patterns onto the drum surface to enhance the visual experience. The drum surface decoration uses wear-resistant materials to ensure stability during long-term use.

[0016] Preferably, the software technology solution includes:

[0017] Interactive Program: Develop interactive software to synchronize the user's drumming action with the system response. The software receives sensor data through the serial port and triggers the image generation algorithm in real time.

[0018] Image generation algorithm: Based on the Stable Diffusion model, combined with the frequency and intensity of the user's drumming, a lion dance pattern with personalized characteristics is generated. The algorithm is optimized to meet the needs of real-time generation.

[0019] QR code generation and sharing: The software integrates a QR code generation module, which encodes the generated lion dance cultural pattern into a QR code, making it convenient for users to scan, save and share it via mobile phones.

[0020] (III) Beneficial effects

[0021] The present invention provides an interactive system for generating cultural arts based on vibration drumming. It has the following beneficial effects: the system uses high-precision sensors and optimized data processing algorithms to improve the response speed and accuracy of drumming interactions, ensuring that every subtle change in the user's actions can be accurately captured and fed back in real time by the system. Through improved communication technology, the stability and efficiency of data transmission are enhanced to ensure high-speed and reliable information transmission even in complex environments. Advanced AIGC technology is used in combination with the Stable Diffusion model to reduce the demand for computing resources for image generation, while improving the speed and quality of image processing, so as to achieve rapid generation of patterns with Chinese cultural characteristics, such as lion awakening cultural patterns. Through user-friendly interactive design, the experience distance between users and culture is shortened, so that users can conveniently experience and learn culture even in non-traditional occasions. Cultural elements are innovatively combined with interactive experience. Through the QR code sharing function, users can easily save and spread the lion awakening patterns they participated in creating, thereby promoting the lion awakening culture.

[0022] The present invention achieves the following technical effects:

[0023] Improve the interactivity of cultural experience: Through the precise capture of drumming devices and sensors, users can experience the lion dance culture in an intuitive and engaging way, which enhances the interactivity between users and culture.

[0024] Enhanced visual experience: Using high-resolution projection technology and customized drum surface decoration, the generated lion dance pattern is vividly displayed in front of the user, providing an immersive visual experience;

[0025] Personalized content generation: Based on the frequency and intensity of the user's drumming, the Stable Diffusion model can generate personalized lion dance patterns to meet the user's demand for personalized experience.

[0026] Improve the system's response speed: Through optimized data processing algorithms and high-speed serial communication technology, the system can quickly respond to the user's drumming actions and achieve real-time image generation;

[0027] Promote cultural dissemination: Through the QR code sharing function, users can easily save and share the lion dance patterns they have participated in creating, effectively promoting the lion dance culture;

[0028] Technical stability and reliability: The use of high-performance server hosts and optimized software algorithms ensures the stability and reliability of the system and reduces experience interruptions caused by technical problems;

[0029] Economic and social benefits: This invention can not only attract more young users to participate in traditional cultural experiences, but also protect and pass on the lion dance culture through digital means, which has a positive impact on the protection of social and cultural diversity. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is the hardware working principle diagram of the present invention;

[0031] Figure 2 It is a schematic diagram of the DFT definition of N-1 of the present invention;

[0032] Figure 3 This is a diagram of the Stable diffusion technology architecture of the present invention;

[0033] Figure 4 It is a schematic diagram of an exhibition item of the present invention;

[0034] Figure 5 It is a structural principle diagram of the present invention;

[0035] Figure 6 It is the formula principle diagram of Fourier transform of x(t) of the present invention;

[0036] Figure 7 This is the principle diagram of the N-1 DFT definition formula of the present invention. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] Example:

[0039] like Figure 1 , 4 As shown in Figure 5, an embodiment of the present invention provides an interactive system for generating cultural arts based on vibrating drums, including a software technical solution and a hardware technical solution working together, the software technical solution and the hardware technical solution include striking the drum surface and a characteristic lion awakening element, the striking drum surface is connected to a server receiving rate through a serial port communication, the server receiving rate is connected to a judgment of the striking frequency and a judgment of the number of striking times, the judging of the striking frequency is connected to an interactive module through a rhythm to form a specific mask, the judging of the number of striking times transmits the number of striking times to Huawei to a corresponding prompt word for text encoding connected to the interactive module, the characteristic lion awakening element is connected to a Stable Diffusion model, and the server receiving rate is connected to a Stable Diffusion model. Diffusion model, StableDiffusion model is connected to the interaction module, the interaction module includes input image encoding and post-processing, the interaction module generates a custom generated image with lion awakening elements, the generated custom generated image with lion awakening elements is connected with the overlay interface design module, the overlay interface design template is connected with the generated QR code, the generated QR code is connected with the object projected on the counter-projection, the input image encoding is connected with the addition of noise, the addition of noise is connected with the text encoding, the text encoding is connected with the sampling process, the sampling process is connected with the UNet network structure, the UNet network structure is connected with the latent space decoding, the post-processing is connected with the control and fine-tuning, and the control and fine-tuning is connected with the iterative optimization;

[0040] Specifically, the hardware technology solution includes:

[0041] Drumming device: A highly sensitive pressure sensor array is installed under the drum surface to capture the force and position of the experiencer's drumming. The sensors are arranged in a grid to ensure full coverage of all areas of the drum surface.

[0042] Sensor and data processing unit: The data collected by the sensor is converted into digital signals through an analog-to-digital converter (ADC) and transmitted to the data processing unit through a high-speed serial communication interface;

[0043] Server host: The server host is equipped with high-performance CPU and GPU to run the Stable Diffusion image generation model, and has powerful data processing and storage capabilities;

[0044] Projector and drum surface decoration: The high-resolution projector projects the interactive interface and generated cultural patterns and lion dance patterns onto the drum surface to enhance the visual experience. The drum surface decoration uses wear-resistant materials to ensure stability during long-term use.

[0045] Software technology solutions include:

[0046] Interactive Program: Develop interactive software to synchronize the user's drumming action with the system response. The software receives sensor data through the serial port and triggers the image generation algorithm in real time.

[0047] Image generation algorithm: Based on the Stable Diffusion model, combined with the frequency and intensity of the user's drumming, a lion dance pattern with personalized characteristics is generated. The algorithm is optimized to meet the needs of real-time generation.

[0048] QR code generation and sharing: The software integrates a QR code generation module, which encodes the generated lion dance cultural pattern into a QR code, making it convenient for users to scan, save and share it via mobile phones.

[0049] Specifically, in the above specific embodiments, the AIGC lion awakening display system designed and studied in the project in this system has a projector and an interactive drum as its main components. The user strikes the drum surface, and the sensor collects the drumming action and sends data to the computer. The program generates corresponding lion awakening content for display based on the user's different striking frequencies and AIGC technology. The combination of interactive array drums and AIGC technology realizes the real-time conversion of user actions to visual content. Embedded development and customized sensing software are used to achieve accurate motion capture and data transmission.

[0050] Hardware system embodiment:

[0051] Drumming equipment:

[0052] Multiple layers of pressure sensors are placed on the inner side of the drum, with each layer detecting the force of the blows in different areas. The sensors are connected via a flexible circuit board to adapt to the curvature of the drum.

[0053] Data processing unit:

[0054] Contains a central processing unit (CPU) and an analog-to-digital converter (ADC) to process the signals from the sensor. The ADC converts the analog signal into a digital signal for further analysis by the CPU.

[0055] Server Host:

[0056] Equipped with high-performance CPU and GPU for running image generation algorithms. The server contains sufficient memory and storage space to support the data processing needs of a large number of concurrent users.

[0057] Projector:

[0058] Short-throw projection technology and alignment technology are used to project the interactive interface and images onto the drum surface.

[0059] Drum decoration:

[0060] Made of durable material and covered with a special coating to improve the visibility of the projected image.

[0061] Software system implementation:

[0062] Interactive Programs:

[0063] Develop event-driven software to monitor changes in sensor data and trigger image generation. Use multi-threading technology to ensure that the software can handle multiple user interaction requests at the same time.

[0064] Image generation algorithm:

[0065] The Stable Diffusion model is used to generate the lion awakening pattern based on the sensor data. The algorithm is optimized to reduce the amount of calculation and increase the speed of image generation.

[0066] QR code generation and sharing module:

[0067] The generated pattern is encoded into a QR code, which users can scan and save with their mobile phones. The module supports social media sharing, making it easy for users to share patterns on different platforms.

[0068] How the hardware works

[0069] When the user hits the drum, the sensor detects the pressure change and converts it into a digital signal through the ADC. The data processing unit receives the digital signal and extracts the force and position information of the hit. The processing unit sends the data to the server host through the serial communication interface. The server host receives the data and triggers the Stable Diffusion model to generate an image. The generated image is projected onto the drum surface through the projector. The software generates a QR code and displays it next to the drum surface through the projector, such as Figure 1 shown.

[0070] Software working steps

[0071] 1. The user starts drumming and the interactive program captures sensor data.

[0072] 2. The program analyzes the data and extracts the characteristics of the strikes, such as force and frequency.

[0073] The frequency is calculated by fast Fourier transform. The Fourier transform of the non-periodic continuous time signal x(t) is as follows: Figure 6 As shown;

[0074] The calculated value is the continuous spectrum of the signal x(t). However, in the actual control system, what can be obtained is the discrete sampling value x(nT) of the continuous signal x(t). Therefore, it is necessary to use the discrete signal x(nT) to calculate the spectrum of the signal x(t). The DFT definition of the finite-length discrete signal x(n), n = 0, 1, ..., N-1 is as follows Figure 2 and 7 As shown;

[0075] Data collection: Use a pressure sensor array to collect pressure signals generated when the user strikes the drumhead.

[0076] Signal preprocessing: Filter the collected analog signal to remove noise and unnecessary frequency components to improve the accuracy of FFT.

[0077] Sampling and quantization use an analog-to-digital converter (ADC) to convert the analog signal into a digital signal in preparation for processing by the FFT algorithm.

[0078] Apply the FFT algorithm: The digitized signal is used as the input of the FFT algorithm, which can convert the time domain signal into the frequency domain, thereby revealing the frequency components of the signal.

[0079] Frequency Analysis: Analyze the FFT results, identify the spectrum of the signal, and find the main frequency components representing the drum hits.

[0080] Frequency Extraction: Extract frequency peaks from the FFT results, which correspond to the main vibration frequencies when the drum is struck.

[0081] Application of results: The extracted frequency information is used to control the parameters of the image generation algorithm. For example, different frequencies correspond to different lion awakening patterns or movements.

[0082] 3. Send feature data to the server and request image generation.

[0083] 4. The server runs the image generation algorithm to generate the lion awakening pattern based on the striking features.

[0084] Stable diffusion technology architecture Figure 3 As shown;

[0085] Input Image Encoding: Use Variational Autoencoder (VAE) to encode the input reference image and convert it into a representation in the latent space.

[0086] Noise Injection: Add noise to the latent space representation obtained by VAE encoding. This step can be regarded as a gradual randomization of the image, providing a basis for the subsequent denoising process.

[0087] Text Encoding: Use CLIPEmbedder or the provided text prompt corresponding to the number of taps to convert it into a text feature vector that matches the latent space.

[0088] Sampling Process: Multiple iterations of sampling are performed depending on the degree of noise addition and the denoising step. In each iteration, the model predicts and removes the noise, gradually recovering the features of the image. This process involves predicting the noise and recovering the image from the noise, which is usually implemented using the DDIM (Denoising Diffusion Implicit Model) sampler.

[0089] UNet Architecture: Use the UNet architecture to predict the noise at each step and perform denoising. UNet is a common convolutional neural network that has an encoder-decoder structure and can process image data.

[0090] Latent Space Decoding: After the sampling process is completed, the VAE decoder is used to convert the final latent space representation back to the image space to generate a new image.

[0091] Post-processing: Perform necessary post-processing on the generated images, such as color correction, contrast adjustment, etc., to improve image quality.

[0092] Control and Fine-tuning: Combine the tapping frequencies extracted by Fast Fourier Transform to form a specific mask to guide specific areas of the image to be repainted or style transferred

[0093] Iterative Refinement: Depending on the quality of the generated results, multiple iterations and refinements may be required to obtain better image results by adjusting sampling parameters, noise levels, or text prompts

[0094] 5. The software receives the generated pattern and displays it on the drum surface through the projector.

[0095] 6. After the user completes the tapping, the software generates a QR code and displays it on the projector for the user to scan.

[0096] The code is as follows:

[0097]

[0098] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An interactive system for generating cultural art based on vibrating drums, comprising software technology solutions and hardware technology solutions working in collaboration, wherein the software technology solutions and hardware technology solutions include drum hitting and characteristic lion awakening elements, and are characterized by: The drum surface is connected to a server receiving rate through serial communication, the server receiving rate is connected to the judging frequency of the knocking and the judging number of knocking, the judging frequency of the knocking is connected to an interactive module through a rhythm to form a specific mask, and the judging number of knocking transmits the number of knocking to Huawei to the corresponding prompt word for text encoding and is connected to the interactive module; The characteristic lion awakening element is connected to a Stable Diffusion model, the server receiving rate is connected to the Stable Diffusion model, and the Stable Diffusion model is connected to the interactive module; The interactive module includes input image encoding and post-processing, the interactive module generates a custom generated image with lion awakening elements, the custom generated image with lion awakening elements is connected to an overlay interface design module, the overlay interface design template is connected to a generated QR code, and the generated QR code is connected to an object projected on the counter-projection.

2. The interactive system for generating cultural art based on vibration drumming according to claim 1, characterized in that: The input image encoding is connected with noise addition, the noise addition is connected with text encoding, the text encoding is connected with a sampling process, the sampling process is connected with a UNet network structure, the UNet network structure is connected with latent space decoding, the post-processing is connected with control and fine-tuning, and the control and fine-tuning are connected with iterative optimization.

3. The interactive system for generating cultural art based on vibration drumming according to claim 1, characterized in that: The hardware technical solution includes: Drumming device: A highly sensitive pressure sensor array is installed under the drum surface to capture the force and position of the experiencer's drumming. The sensors are arranged in a grid to ensure full coverage of all areas of the drum surface. Sensor and data processing unit: The data collected by the sensor is converted into digital signals through an analog-to-digital converter (ADC) and transmitted to the data processing unit through a high-speed serial communication interface; Server host: The server host is equipped with high-performance CPU and GPU to run the Stable Diffusion image generation model, and has powerful data processing and storage capabilities; Projector and drum surface decoration: The high-resolution projector projects the interactive interface and generated cultural patterns and lion dance patterns onto the drum surface to enhance the visual experience. The drum surface decoration uses wear-resistant materials to ensure stability during long-term use.

4. The interactive system for generating cultural art based on vibration drumming according to claim 1, characterized in that: The software technology solution includes: Interactive Program: Develop interactive software to synchronize the user's drumming action with the system response. The software receives sensor data through the serial port and triggers the image generation algorithm in real time. Image generation algorithm: Based on the Stable Diffusion model, combined with the frequency and intensity of the user's drumming, a lion dance pattern with personalized characteristics is generated. The algorithm is optimized to meet the needs of real-time generation. QR code generation and sharing: The software integrates a QR code generation module, which encodes the generated lion dance cultural pattern into a QR code, making it convenient for users to scan, save and share it via mobile phones.