Virtual piano based on 3D depth camera module and method for realizing same

Abstract

The invention discloses a virtual piano based on 3D depth camera module and a method for realizing same. The virtual piano comprises a projection device, a 3D depth camera module, a player, a communication module and a processing module. The projection device is used for projecting the piano key pattern and transmitting the projected key position parameters to the processing module through the communication module. The 3D depth imaging module is used for collecting the depth image of the user's hand in real time and transmitting it to the processing module through the communication module. Theplayer is used for playing music tones; The processing module is used for analyzing the depth information of the hand according to the depth image of the hand, recognizing the knocking action of thehand, judging the knocking key situation according to the key position parameters, and controlling the player to play the corresponding music tone of the key according to the knocking key situation. The virtual piano can accurately identify the movement of the finger skeleton of the user, accurately judge the position of the finger percussion, play the corresponding music sound, smoothly simulatethe piano playing, and also can bring good interactive experience.

Description

technical field [0001] The invention relates to 3D depth camera applications, in particular to a virtual piano based on a 3D depth camera module and an implementation method thereof. Background technique [0002] In life, physical pianos are bulky and expensive, and for beginners, the early learning process is very boring and difficult, lacking in fun; while the simulated pianos on some mobile phone applications are small in size and portable, but are limited by the size of the screen , is far from a professional piano, and it is difficult to play smoothly or simulate piano practice. [0003] Virtual piano projection device (CN205594300U) discloses a device for projecting a piano key pattern to a plane, detecting that the user operates on the projected key pattern and playing the corresponding scale sound. 2D laser, covering the surface of the piano key pattern, the camera captures the laser spot formed by the user's operation and judges the position of the spot, so as to d...

AI Technical Summary

Problems solved by technology

[0005] 1. Large error in judging the position of the finger: This technical solution uses the laser spot pattern to identify. When playing, the palm is naturally placed on the table for support. When the finger is lifted, the laser will also generate a spot on the palm. The distance of the spot is judged by the 2D image. It is easy to make errors, so it is impossible to accurately identify the button action, and the playing experience is poor;

[0006] 2. There is a large error in judging finger movements: different people have different finger sizes, and it is easy to misjudgment at the critical position of tapping and releasing, and the gesture habits of tapping are different, and the position of hands is not uniform, and it is easy to cause false touch;

[0007] 3. Low interactivity: unable to accurately record the entire operation process of the user for evaluation, feedback or interaction

Images