Musical toy using mobile terminal device

Abstract

To provide a music playing toy which detects rotational speed of a rotation gesture by the rotation gesture on a touch panel by one contact point of an electronic device such as a portable terminal device, determines a playing position of a musical score inputted from digital musical score input means on the basis of the detected rotational speed, and plays a music instrument on the basis of the determined playing position or to provide a music playing toy which detects a virtual radius of rotation, determines a sound volume on the basis of the detected virtual radius of rotation and plays the music instrument on the basis of the detected sound volume.SOLUTION: Detection means for detecting rotational speed of a rotation gesture on a touch panel by one contact point of an electronic device such as a portable terminal device and determination means for determining a playing position of a musical score on the basis of the detected rotational speed are provided, and a music instrument is played on the basis of the determined playing position, or detection means for detecting a virtual radius of rotation of the rotation gesture and determination means for determining a sound volume on the basis of the detected virtual radius of rotation are provided, and the music instrument is played on the basis of the determined sound volume.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a musical playing toy that uses a mobile terminal device such as a smartphone. [Background technology]

[0002] Conventionally, as shown in Patent Document 1, for example, a musical score is formed by printing and recording lines corresponding to musical notes on a roll of paper or the like, and the movement of the roll of paper or the like is photographed, recorded, or digital video data is created by other methods. An image input means is provided to continuously extract and input still images from the still images, and an image processing means is provided to detect the recorded musical notes from the input still images by image processing. The detected musical notes are then used to play the musical score by making the sounds of the corresponding notes. This musical toy simulates an automatic piano.

[0003] According to the document, the image input to the image input means is not limited to video input from a camera, but may also be a video in MP4 format or the like that captures the process of winding up a roll of paper with musical notes or the like printed on it, or the process of winding up a roll of paper on an actual player piano as it is played, or a means for inputting similar electronically created video.

[0004] Furthermore, Patent Documents 2, 3, and 4 disclose a function to play music based on separately prepared digital music score data, such as MIDI data, using a piano roll that electronically simulates the musical score and performance recorded on roll paper for an automatic piano.

[0005] Furthermore, Patent Document 5 discloses a technology for receiving motion information corresponding to the operator's movements transmitted from an operating terminal and editing existing digital sheet music data based on the received motion information. Although digital sheet music data is called performance data in Patent Document 5, it will be referred to here as digital sheet music data (score data for short) to contrast it with sheet music images, which will be discussed later. Furthermore, digital sheet music images and digital sheet music data will be collectively referred to as digital sheet music.

[0006] On the other hand, Patent Documents 6 and 7 disclose technologies that recognize the operation of touching and moving multiple fingers on the touch panel of a mobile terminal device as a gesture. These technologies recognize the rotation of fingers on the panel as a gesture event that occurs at regular intervals, and send the recognized information to the browser to dynamically rotate elements on a web page.

[0007] Furthermore, Patent Document 8 discloses a technique for recognizing the rotation of a finger on a touch panel, in which a virtual circle is set on a position coordinate system, and the change over time in at least one of the center coordinate of the virtual circle, the radius of the virtual circle, and the rotation angle of the virtual circle is calculated as an operation signal.

[0008] Here, as an input image for the device described in Patent Document 1, consider a still image of an electronically created musical score, which is not described in the same document. This is an electronic image of the musical score recorded on roll paper described in Patent Document 1, saved in a file format such as PNG or JPEG, and will be called a digital musical score image, or musical score image for short. Using this musical score image, the musical score image can be cut out by sequentially reading it out at regular intervals while shifting the position, in the same way as cutting out still images from camera or video data described in Patent Document 1, and then passing the cut-out images to image processing means and performing the same processing as described in Patent Document 1, thereby making it possible to play the musical score image.

[0009] In this case, if the position from which the still image is cut out as the playing position of the musical score is determined based on the speed of rotation obtained by the rotation gesture on the touch panel of the mobile terminal device, it becomes possible to change the playing speed in a manner that depends on the rotation speed of the rotation gesture.

[0010] Furthermore, when recognizing the rotation of a finger on the touch panel as a gesture, the virtual diameter of the rotation circle is recognized and associated with the volume of the performance to produce the sound of a note, thereby enabling adjustment of the volume during performance, which was not possible with the device shown in Patent Document 1.

[0011] Furthermore, if the digital music score data (MIDI data, etc.) described above is used as the music score instead of the music score image, a simpler device can be realized that does not require image processing means.

[0012] However, Patent Documents 2, 3, 4, and 5 do not disclose a technique for changing the actual playing speed in real time, and Patent Documents 6 and 7 disclose a method for recognizing the angle and speed of finger rotation by gesture, but do not disclose a method for determining the playing position on the score for playing the score.

[0013] Furthermore, Patent Document 8 discloses a method for determining the radius and rotation angle of a virtual circle by using two or more fingers without determining the center of the circle, and by using one finger with the center of the circle determined in advance, but does not disclose a method for determining the diameter of a virtual circle by using one finger without determining the center of the circle in advance. [Prior art documents] [Patent documents]

[0014] [Patent Document 1] Patent application 2020-210850 [Patent Document 2] Patent Publication No. 2001-51586 [Patent Document 3] Patent Publication No. 2007-279470 [Patent Document 4] Patent Publication No. 2015-069151 [Patent Document 5] Patent Publication No. 2004-053930 [Patent Document 6] Patent Publication No. 2019-57295 [Patent Document 7] Patent Publication No. 2016-201122 [Patent Document 8] Patent No. 6058118 Summary of the Invention [Problem to be solved by the invention]

[0015] The problem to be solved by this invention is to provide a practical method and apparatus for detecting the rotation speed of a gesture from a rotation operation on a touch panel that is recognized as a gesture event by a single contact point in an electronic device such as a mobile terminal device, determining a performance position on a musical score to be performed from the detected rotation speed, and performing the musical score based on the determined performance position.

[0016] Another problem that the present invention aims to solve is to provide a practical method and device that allows for volume adjustment by gesture during performance in an electronic device such as a mobile terminal device, by calculating the virtual diameter of the rotation circle of the gesture from a rotation operation on a touch panel that is recognized as a gesture event made by a single contact point, and determining the volume of the performance based on that length. [Means for solving the problem]

[0017] The most important feature of the present invention is that it comprises a score reading means for reading digitally recorded score to be used in performance, a gesture detection means for recognizing a rotation gesture on a touch panel made by a single contact point, a rotation speed detection means for determining the rotation speed of the gesture from touch position coordinates on a two-dimensional coordinate system obtained at regular time intervals from the gesture detection means, and a performance position determination means for determining a performance position on the score from the rotation speed of the gesture detected by the rotation speed detection means, and performing the score based on the determined performance position.

[0018] Another major feature of the present invention is the provision of gesture detection means for recognizing a rotation gesture on a touch panel made by a single contact point, diameter detection means for determining the diameter of a virtual circle of rotation from touch position coordinates on a two-dimensional coordinate system obtained at regular time intervals from the gesture detection means, and volume determination means for determining the volume of the performance from the determined diameter of the virtual circle, and the performance is performed at a volume determined by the volume determination means. [Effects of the Invention]

[0019] According to the present invention, a performance position determination means is provided that determines the performance position of the musical score from the rotation speed of a rotation gesture made by one contact point on the touch panel, and the musical score is played based on the determined performance position, making it possible to change the performance speed of the musical score according to the rotation speed of the gesture, thereby offering the advantage that it is possible to provide functionality equivalent to that of a conventional musical performance toy equipped with a physical hand-crank device using only an electronic device such as a mobile terminal device.

[0020] Furthermore, according to the present invention, a diameter detection means is provided for determining the diameter of the virtual circle rotated by a rotation gesture made by one contact point on the touch panel, and a volume determination means is provided for determining the volume of the performance from the determined diameter of the virtual circle, and by changing the volume of the performance based on the determination by the volume determination means, there is an advantage in that a function not available in conventional musical playing toys can be provided in which the volume during performance can be freely changed by gesture. [Brief explanation of the drawings]

[0021] [Figure 1] FIG. 1 is an explanatory diagram showing the first embodiment. (Example 1) [Figure 2] FIG. 2 is an explanatory diagram showing the hardware and software configuration in the first embodiment. [Figure 3] FIG. 3 is a flowchart showing the operation of the software in the first embodiment. [Figure 4] FIG. 4 is an explanatory diagram 1 for explaining the contents of gestures in the first embodiment. [Figure 5] FIG. 5 is an explanatory diagram 2 for explaining the contents of gestures in the first embodiment. [Figure 6] FIG. 6 is an explanatory diagram for explaining a typical musical score. [Figure 7] FIG. 7 is an explanatory diagram for explaining a digital music score image in the first embodiment. [Figure 8] FIG. 8 is an explanatory diagram showing a second embodiment of the method. (Example 2) DETAILED DESCRIPTION OF THE INVENTION

[0022] The objective of realizing a musical toy simulating a conventional player piano using only an electronic device such as a mobile terminal device is realized by inputting a digital musical score from a digital musical score input means, detecting the speed of rotation caused by a gesture made with a single touch on the touch panel of the mobile terminal, determining a performance position on the musical score from the detected speed of rotation, and performing the musical score based on the determined performance position, thereby performing the musical score according to the speed of rotation of the gesture.Furthermore, the objective of realizing volume adjustment by gestures when playing the musical toy is realized by calculating the diameter of a virtual circle rotated by a gesture made with a single touch on the touch panel, and determining the volume of the performance from the calculated diameter of the virtual circle. [Example]

[0023] FIG. 1 is an overall diagram of a first embodiment of the device of the present invention, with 100 representing the smartphone itself and 101 representing the image output unit using a liquid crystal panel. The liquid crystal panel is a touch panel and also serves as the information input unit 102 operated by the user. The area indicated by the dashed frame 1011 is the score image display unit. The score image is read from the music score input device within the smartphone, and the necessary portion is cropped or rendered to display the score image in real time as the performance progresses. This simulates the glass window on a conventional player piano through which the roll paper is rewound as the performance progresses. In this embodiment, the score image displayed moves in the direction of the arrow 302 as the performance progresses. The dashed line 1020 is a virtual timing line indicating the playing position, indicating that sound will be produced when the note is on that line. 2001, 2002, and 2003 are notes in the score image displayed on the liquid crystal panel 101. Reference numerals 2010 and 2011 denote marker lines recorded on both ends of the musical score image for detecting the position of notes displayed on the panel 101. In this embodiment, these marker lines are used by the image processing means described later, but are not essential for producing the sound of notes using the digital musical score. The portion enclosed by the dashed line in 1012 is the keyboard display section. When a note is present at the performance position, the color of each white key and black key changes to gray to indicate that the corresponding key has been pressed. Furthermore, when the note that was at the performance position moves away over time and is no longer present, the color of each white key and black key changes back to its original color to indicate that the corresponding key has been released. Reference numeral 300 denotes a finger used to perform gesture operations on the LCD panel 101, and 301 denotes an arrow that simulates the rotation gesture made by finger 300. When a finger 300 is rotated in the direction of an arrow 301, the musical score moves in the direction of an arrow 302 according to the rotation speed, and when a note comes on a virtual timing line 1020, which is the performance position, the note is sounded, and when the note moves off the timing line 1020, the sound is erased, thereby realizing the performance.

[0024] FIG. 2 is a diagram showing the hardware and software configuration of this embodiment of the device of the present invention. Reference numeral 1 denotes a CPU (Central Processing Unit), which performs various processes and issues commands to various means according to programs stored in nonvolatile memory (not shown), thereby realizing the present invention. Reference numeral 2 denotes a musical score image input means, which the CPU 1 uses to read musical score images for performance purposes from a musical score storage means (not shown) within the device itself, which is comprised of nonvolatile memory or the like, or from an external website. The musical score images stored in the musical score storage means can also be created from musical score data such as MIDI. In this case, the musical score data must be separately converted into musical score images and stored in the musical score storage means; details of this are omitted. Reference numeral 7 denotes a gesture detection means, which recognizes a rotation gesture made by a finger 300 on the touch panel 102 and detects the position coordinates (x, y) of the finger 300 on the touch panel 102 at regular intervals. From the detected position coordinates (x, y) of finger 300, CPU 1 calculates the speed of the rotation caused by the gesture using rotation speed detection means 8 configured by a software program, and from this determines the performance position on the musical score using performance position determination means 10, also configured by a software program. Also, from the detected position coordinates (x, y) of finger 300, CPU 1 calculates the diameter of the virtual circle of rotation caused by the gesture using virtual circle of rotation diameter detection means 9 configured by a software program, and from this determines the volume of the performance using volume determination means 11, also configured by a software program. 3 is image output means configured by a liquid crystal panel 101, etc. CPU 1 trims the musical score image input from musical score image input means 2 to a size required for display on musical score image display unit 1011 based on the performance position determined by performance position determination means 10, and passes the trimmed image to image output means 3, and image output means 3 displays the musical score image passed from CPU 1 on musical score image display unit 1011. The CPU 1 also cuts out a portion corresponding to the playing position from the score image read from the score image input means 2 and sends it to the image processing means 4, and the image processing means 4 specifies the position of the note from the cut-out score image passed from the CPU 1 by image processing and detects whether or not the note is present at the specified position.Having detected whether or not a note is present, the image processing means 4 returns this information to the CPU 1. The CPU 1 then reads from the keyboard state storage means 5 the state of the keyboard corresponding to the note at the time the previous still image was received. If the previous state of the key was "not pressed" and it is determined that a note is present in the currently received still image, it determines to play the corresponding note. Similarly, if the keyboard state at the previous processing time was "pressed" and it is determined that no note is currently present, it determines to stop the corresponding note from playing, and rewrites the contents of the keyboard state storage means 5 to the latest state. Based on this determination, the CPU 1 then instructs the sound output means 6 to play or stop the corresponding note. If the note is to be played, the instruction is given together with the volume determined by the volume determination means 11. The sound output means 6 has the function of playing or stopping the sound of the corresponding note in a preset tone at a specified volume in accordance with the instructions from the CPU 1, thereby allowing music to be played according to the musical score image. Furthermore, the CPU 1 instructs the image output means 3 to perform a process of changing the color of the corresponding key according to the decision to play or stop the sound of the corresponding note, and the image output means 3 changes the color of the corresponding key on the keyboard display section 1012 according to the instruction.

[0025] Figure 3 is a flowchart showing the software operation of this embodiment of the device of the present invention. First, in S1, CPU 1 waits for a gesture event to be detected by gesture detection means 7. In this embodiment, when a gesture operation using finger 300 begins, an event occurs at intervals of 1 / 60 seconds. The position on the panel where finger 300 first touches is set as origin O: (x, y) = (0, 0), and relative coordinates from the origin are detected every time until finger 300 is released from the panel. If a gesture is detected, the process proceeds to S2, where rotation speed detection means 8 and virtual circle diameter detection means 9 calculate the rotation speed and virtual circle diameter caused by the gesture, respectively.

[0026] Here, we will explain how to determine the rotation speed and the diameter of the virtual circle using the speed detection means 8 and the virtual circle diameter detection means 9 from the position coordinates of the finger 300 on the panel due to a gesture event. Figure 4 shows the coordinate system on the panel and the feature points in gesture operations. In this embodiment, the position where the finger 300 first touches is set as the origin O: (x, y) = (0, 0), with the positive x-axis on the right and the positive y-axis on the bottom. Dh and Dv represent the distances between feature points Plc and Prc, and Ptc and Pbc, respectively, which are used to determine the diameter of the virtual circle of rotation.

[0027] 5 is a graph showing the transition of coordinates (x, y) over time when finger 300 rotates due to a gesture detected by gesture detection means 7. The horizontal axis represents the number of events that occurred (0 to 119), and the vertical axis represents the coordinate value, with the solid and dashed lines representing the x and y values ​​when the events occurred, respectively.

[0028] From these, if the coordinate values ​​of the most recent event are (X, Y) and the coordinate values ​​of the event that occurred just before that are (Xp, Yp), then the rotation speed V is:

number

[0029] On the other hand, the diameter of the virtual circle of rotation is roughly equivalent to the distance between the characteristic points where the slope of the curve representing the transition of the x coordinate or the curve representing the transition of the y coordinate in Figure 5 changes from negative to positive (represented by upward-pointing black and white triangles Plc and Ptc in the figures) and the characteristic points where the slope changes from positive to negative (represented by downward-pointing black and white triangles Prc and Pbc in the figures). If these are respectively Dh and Dv, and these two values ​​are generally different, the average of the two will be taken as the diameter of the virtual circle. Therefore, if the coordinate values ​​of Plc are (Xlc, Ylc), the coordinate values ​​of Prc are (Xrc, Yrc), the coordinate values ​​of Pbc are (Xbc, Ybc), and the coordinate values ​​of Ptc are (Xtc, Ytc), and the diameter of the virtual circle is D, then

number

[0030] The above calculations are performed by the rotation speed detection means 8 and the virtual circle diameter detection means 9, respectively, and once the rotation speed and the diameter of the virtual circle caused by the gesture operation with the finger 300 are determined, the CPU 1 determines the performance position and volume by the performance position determination means 10 and the volume determination means 11 in S3.

[0031] Here, we will explain the digital representation of a musical score. Figures 6 and 7 illustrate how a typical musical score is represented and played as a digital musical score image. Figure 6 shows a typical musical score, with 2004, 2005, 2006, 2007, and 2008 representing quarter notes of different scales. The symbol 400 is a tempo symbol, indicating that the score should be played at a speed of 80 quarter notes per minute. Figure 7 shows the beginning of a digital musical score image obtained by converting the musical score shown in Figure 6 into a digital image. The coordinates are plotted with the positive x-axis on the right and the positive y-axis on the top. The notes included in the circle indicated by 401 in Figure 6 are indicated as 2004, 2005, 2006, 2007, and 2008. 2010 and 2011 are marker lines recorded at both ends of the digital musical score image to detect the positions of the notes. In this embodiment, the width of the musical score image is 384 pixels, the width of a note is 2 pixels, the gap between adjacent notes is 1 pixel, and since 85 notes (= 7 octaves [1 octave is 12 notes including semitones] + 1 note) can be played, the distance between marker lines 2010 and 2011 is 256 pixels = (2 + 1) × 85 + 1 (+1 is the gap between the marker lines). Note that in this embodiment, marker lines 2010 and 2011 are placed on the musical score image to detect the positions of the notes, but if the position on the x-axis of a given note is determined in advance, these marker lines 2010 and 2011 are not essential.

[0032] In this embodiment, quarter note 2004 in Fig. 6 is located three octaves above the lowest note, and therefore corresponds to the 36th note counting from the lowest note (=12 notes x 3 octaves). Therefore, there is a distance of 109 pixels = (2 + 1) x 36 + 1 between quarter note 2004 and marker line 2010 in Fig. 7. Similarly, there are distances of 115, 121, and 124 pixels between quarter notes 2005, 2006, and 2007 and marker line 2010, respectively.

[0033] 7, the length in the y-axis direction of quarter notes 2004, 2005, 2006, 2007, and 2008 corresponds to the length of a quarter note, and in this embodiment, the distance from the start of quarter note 2004 to the start of quarter note 2005 is 180 pixels. Similarly, the distance between 2005 and 2006, 2006 and 2007, and 2007 and 2008 is also 180 pixels. Although not shown here, for an eighth note, the length is half that, 90 pixels, for a sixteenth note, 45 pixels, and for a half note, 360 pixels.

[0034] The performance of the musical score in Figure 6 progresses from left to right, as indicated by arrow 500, following the musical score. On the other hand, the performance of the musical score image progresses from bottom to top, as indicated by arrow 500 in Figure 7. Dashed line 501 in Figures 6 and 7 indicates the performance position. By moving this dashed line in the direction of arrow 500 at regular time intervals, the performance on the musical score and the musical score image progresses. Therefore, the distance that dashed line 501 moves is determined based on the rotation speed at the time a gesture event occurs (every 1 / 60 second in this embodiment). For example, in this embodiment, the tempo marking 400 in Figure 6 indicates a performance speed of 80 quarter notes per minute, so on the digital musical score image in Figure 7, the movement should be 14,400 pixels (= 180 pixels × 80 times) per minute. Gesture events occur at intervals of 1 / 60 seconds, so for each gesture event, the rotation speed V calculated in step S2 is known to have an average value of approximately 15.0 based on actual measurements, so if we calculate the constant C so that the number of pixels is 4 when the speed V is 15.0, then 4 pixels = 15.0 × C, so C = 4 / 15.0 = 0.2667. Therefore, if the number of pixels moved when the rotation speed is V is N, then

number

[0035] On the other hand, the volume is determined by the volume determination means 11 as follows. In this embodiment, it is known that the diameter D of the virtual circle detected by the virtual circle of rotation diameter detection means 9 is actually measured and falls within the range of approximately 0 to 236. Therefore, the detected value is clipped between 0 and 236 and scaled to the range of volume that can be specified to the sound output means 5. In this embodiment, the value is 0 to 127, so if the volume specified to the sound output means 5 is Vol, then

number

[0036] After determining the playing position and volume in step S3, the process proceeds to step S4, where the score image is trimmed based on the playing position so that it can be displayed in the score display area 1011 on the LCD panel 101. For example, if the playing position is the position indicated by dashed line 501 in FIG. 7, the image is trimmed so that the area enclosed by dashed line 502, 384 pixels high and 384 pixels wide, with this line centered vertically, is displayed in the score display area 1011. The trimmed score image is passed to the image output means 3, and the CPU 1 instructs the image to be displayed in the score display area 1011 of the LCD panel 101. Furthermore, in step S5, the CPU 1 cuts out a portion of the score image necessary for image processing based on the playing position determined in step S3. In this embodiment, the portion necessary for image processing is the image of one line above dashed line 501, which indicates the playing position in FIG. 7. In step S6, the CPU 1 passes the score image cut out in step S5 to the image processing means 4, and instructs the image processing means 4 to detect musical notes. In S7, the image processing means 4 identifies the position of the note from the extracted score image, detects whether or not a note is present at the identified position, and returns the result to the CPU 1. In S8, the CPU 1 determines whether a new key has been pressed or released based on the note detected from the digital score image and the information stored in the keyboard status storage means 5 indicating whether the key corresponding to the note is pressed or released based on the previously input digital score image, and updates and stores the status in the keyboard status storage means 5. In S9, based on the determination of whether a new key has been pressed or released in S8, the CPU 1 instructs the sound output means 6 to either play or stop the sound of the note corresponding to that key at the volume determined in S3. In S10, based on the determination of whether a new key has been pressed or released in S8, the CPU 1 instructs the image output means 3 to change the color of the corresponding key. This completes the series of processes associated with the occurrence of a gesture event, and the process returns to S1 to wait for the gesture detection means 7 to detect a gesture event.

[0037] The image processing performed by the image processing means 4 in this embodiment to detect notes from a digital sheet music image is as described in Japanese Patent Application No. 2020-210850. In this embodiment, marker lines 2010 and 2011 are placed on the digital sheet music image to determine the x-coordinate position of the notes. This allows notes to be detected from a digital sheet music image in exactly the same manner as described in Japanese Patent Application No. 2020-210850. However, unlike the configuration described in Japanese Patent Application No. 2020-210850, in which an actually photographed image is input, this embodiment uses a digitally created sheet music image as input. This eliminates the need to consider camera shake during photography or fluctuations in the roll paper during hand-turned performance. Therefore, as previously mentioned, if the position of the notes on the x-axis of the sheet music image is predetermined, the marker lines 2010 and 2011 are not required. Equivalent processing can be achieved by examining the brightness level of the pixel at the predetermined position or the difference in brightness level between the pixel and adjacent pixels.

[0038] As described above, this embodiment provides a musical score input means for inputting a musical score image, a gesture detection means for recognizing a rotation gesture made by a single contact point on the touch panel, a rotation speed detection means for determining the rotation speed of the detected gesture, and a performance position determination means for determining a performance position on the musical score image from the rotation speed of the gesture detected by the rotation speed detection means. By adopting a method for playing a digital musical score based on the determined performance position, it is possible to realize a musical playing toy that simulates a conventional automatic piano using only an electronic device such as a mobile terminal device.

[0039] The present invention also provides a gesture detection means for recognizing a rotation gesture made by a single contact point on the touch panel, a virtual circle of rotation diameter detection means for determining the diameter of the virtual circle of rotation of the detected gesture, and a volume determination means for determining the volume of the notes from the detected diameter of the virtual circle, and by changing the volume of the notes to be played based on the determination of the volume determination means, it is possible to realize functions that were not possible with conventional musical playing toys that simulate automatic pianos.

[0040] In this embodiment, the description has been given of a rotation gesture with one contact point by the finger 300 on the touch panel, but as long as the gesture can be recognized, it may be a gesture with one contact point using, for example, a touch pen.

[0041] FIG. 8 shows a second embodiment of the present invention. In the first embodiment, a score image is input from the score image input means 2, but in this embodiment, this is changed to a score data input means 12 that inputs digital score data such as MIDI instead of a score image.

[0042] In this embodiment, since there is no musical score image input means, the musical score image displayed on the musical score display unit 1011 is created by the image generation means 14 from the playing position determined by the playing position determination means 9 and the musical score data input from the musical score data input means 12, and then an image for display on the musical score display unit 1011 is created and displayed; however, details of this will be omitted here.

[0043] In addition, in the first embodiment, the portion corresponding to the playing position was extracted from the score image read from the score image input means 2 and sent to the image processing means 4, which then identified the position of the note and detected whether or not there is a note at the identified position. However, in this embodiment, the note is determined and played using the score data analysis means 13, which directly detects whether or not there is a note at the playing position based on the score data input from the score data input means 12 and the playing position determined by the playing position determination means 10.

[0044] As noted in Patent Document 2, musical score data generally stores data indicating which notes are to be played or stopped at which performance positions. For example, in the case of MIDI data, data is stored in groups called tracks in the form of repeating delta times (durations, as defined in Patent Document 2) and events (note events, as defined in Patent Document 2). Delta times are calculated using a separately specified quarter note length and time unit, and represent the elapsed time from the previous specified delta time. Events specify the scale, volume, and on / off status of the current note. Therefore, once a performance position is determined by the performance position determination means 10, the CPU 1 inputs and analyzes musical score data from the musical score data input means 12 using the musical score data analysis means 12, which is implemented as a software program, to determine which notes have changed between the performance position at the time of the previous gesture event and the current performance position. Once the changed notes have been determined, the sound output means 6 is instructed to play or stop the sound, along with the volume determined by the volume determination means 11, thereby achieving functionality equivalent to that of Example 1.

[0045] According to this embodiment, the musical score image input means 2 is replaced by a musical score data input means 12, and instead of using image data for the musical score to be played, musical score data is used, so that the musical notes are not determined by image processing, and therefore a music playing toy can be realized more easily. [Industrial Applicability]

[0046] Instead of gestures, the speed of rotation is detected using electrical signals to determine the playing position, and the musical score is played based on the determined playing position, making this system applicable not only to mobile terminal devices, but also to personal computer-based devices and standalone musical playing toys. [Explanation of symbols]

[0047] 1 CPU (Central Processing Unit) 2. Music score image input method 3 Image output means 4 Image processing methods 5. Keyboard status storage means 6 Sound output means 7 Gesture detection method 8. Rotational speed detection means 9. Means for detecting the diameter of the virtual circle of rotation 10 Performance position determining means 11 Volume determination means 12 Music score data input means 13 Music score data analysis method 14 Image generation means

Claims

1. A musical performance toy comprising: a digital music score input means; a gesture detection means for recognizing a rotation operation on a touch panel by a single contact point as a gesture; a means for detecting the rotation speed of the gesture from the detected gesture; and a means for determining a performance position of the digital music score input from the digital music score input means and the detected rotation speed, wherein the musical note to be played is determined based on the determined performance position and a performance is performed; the musical performance toy further comprising: a means for detecting the diameter of the virtual circle of the gesture from the same gesture as that detected by the gesture detection means, independently of the detection of the rotation speed; and a volume determination means for determining the volume of the note from the detected diameter of the virtual circle, wherein the musical performance is performed by changing the volume of the note to be played based on the determination of the volume determination means.

2. 2. The musical playing toy according to claim 1, wherein the digital musical score is read as a digital musical score image from a musical score image input means based on the determined playing position, and the notes to be played are determined by an image processing means and then played.

3. 2. The musical playing toy according to claim 1, wherein the digital musical score is read as digital musical score data from a musical score data input means, and musical notes to be played are determined by a musical score data analysis means based on the read digital musical score data and the determined playing position, and then the musical score is played.

Images