Music playing method and device using blocks

Abstract

Disclosed are a music playing method and device using blocks. A music playing method using at least one block, the method being performed by a music playing device according to an embodiment of the present invention may comprise the steps of: capturing an image including the at least one block through a reception device mounted on the music playing device; recognizing the captured at least one block by using a trained deep learning neural network model; determining an arrangement structure of the recognized block; and playing the music in the music playing device on the basis of the determined arrangement structure.

Description

Method and device for playing music using blocks

[0001] The present invention relates to a method and device for playing music using blocks. More specifically, the present invention relates to a method and device for recognizing the arrangement of blocks using an artificial intelligence model and playing corresponding music using a photographing device mounted on a music playback device.

[0002] Recent technological advancements have made software coding education mandatory starting in elementary school. Consequently, coding toys (e.g., unplugged learning tools) for beginners are gaining attention. Meanwhile, special education schools are also utilizing various coding toys for students with disabilities (e.g., visual impairment, borderline intellectual disability, etc.). However, due to methods and difficulty levels that are not suitable for these students, effective coding education is hindered.

[0003] Accordingly, there is a need for an educational method that can efficiently provide coding education to visually impaired or non-disabled people by simply using a simple arrangement of coding toys (e.g., blocks).

[0004] The technical problem of the present invention is to provide a method and device for playing music using blocks.

[0005] Another technical task of the present invention is to provide a method and device for recognizing a block arrangement structure using a photographing device and playing music using an artificial intelligence neural network model.

[0006] Another technical task of the present invention is to provide a music reproduction method and device that matches a coding algorithm to a musical composition.

[0007] The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the present invention belongs from the description below.

[0008] According to one aspect of the present invention, a method for playing music using at least one block performed by a music playing device may include a step of photographing an image including the at least one block through a receiving device mounted on the music playing device, a step of recognizing the photographed at least one block using a learned deep learning neural network model, a step of determining a layout structure of the recognized block, and a step of playing the music on the music playing device based on the determined layout structure.

[0009] In the music reproduction method according to the present invention, the receiving device may be a photographing device.

[0010] In the music reproduction method according to the present invention, the deep learning neural network model may be a convolutional neural network model.

[0011] In the music reproduction method according to the present invention, the step of determining the arrangement structure may be a step of determining the coordinate values ​​of the position where at least one recognized block is placed and the block type.

[0012] In the music reproduction method according to the present invention, the learned deep learning neural network model may be characterized in that the arrangement structure of the blocks is determined as a start block that plays at least one or more connected instrument blocks once, a repeat block that repeatedly plays at least one or more connected instrument blocks after music of at least one or more instrument blocks connected to the start block is played, a level block that specifies a performance level of the instrument blocks, and an arrangement structure in which at least one or more of the instrument blocks corresponding to each instrument for the music reproduction is interconnected.

[0013] In the music reproduction method according to the present invention, the music reproduction device can reproduce the music based on all of the plurality of arrangement structures recognized through the receiving device.

[0014] According to another aspect of the present invention, a music playback device using blocks may be provided. The device includes a receiving device that captures an image including at least one block, a memory, and at least one processor, wherein the processor recognizes the at least one block captured by the receiving device using a trained deep learning neural network model, determines the arrangement structure of the recognized block, and plays the music based on the determined arrangement structure.

[0015] According to another aspect of the present invention, software or a computer-readable non-transitory recording medium having executable instructions for performing a music reproduction method may be provided.

[0016] According to another aspect of the present invention, a computer-readable recording medium storing a program for performing a music reproduction method may include a step of photographing an image including at least one block through a receiving device mounted on the music reproduction device, a step of recognizing the photographed at least one block using a learned deep learning neural network model, a step of determining a layout structure of the recognized block, and a step of reproducing the music on the music reproduction device based on the determined layout structure.

[0017] The features briefly summarized above of the present invention are merely exemplary aspects of the detailed description of the present invention described below and do not limit the scope of the present invention.

[0018] According to the present invention, a music reproduction method and device can be provided that can provide direct / indirect coding education by simply assembling blocks.

[0019] In addition, according to the present invention, a music reproduction method and device for arranging and executing music according to coding logic can be provided.

[0020] In addition, according to the present invention, not only the visually impaired but also the non-disabled can easily access coding, and can learn coding through synesthetic elements (tactile, auditory, etc.) rather than visual elements.

[0021] The effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by a person having ordinary skill in the art to which the present invention belongs from the description below.

[0022] FIG. 1 is a drawing for explaining a music playback method using blocks according to one embodiment of the present invention.

[0023] FIG. 2 is a drawing for explaining the types of blocks used in a music reproduction method according to one embodiment of the present invention.

[0024] FIG. 3 is a drawing for explaining the types of blocks used in a music reproduction method according to one embodiment of the present invention.

[0025] FIG. 4 is a drawing for explaining a method of arranging blocks for music playback according to one embodiment of the present invention.

[0026] FIGS. 5 to 10 are drawings for explaining connection rules of blocks for music playback according to one embodiment of the present invention.

[0027] FIG. 11 is a drawing for explaining the configuration of a music playback device utilizing blocks according to one embodiment of the present invention.

[0028] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings so that those skilled in the art can easily implement the invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein.

[0029] In describing embodiments of the present invention, if a detailed description of a known configuration or function is judged to obscure the gist of the present invention, a detailed description thereof will be omitted. Furthermore, portions irrelevant to the description of the present invention in the drawings have been omitted, and similar portions have been designated with similar drawing reference numerals.

[0030] In the present invention, when a component is said to be "connected," "coupled," or "connected" to another component, this may include not only a direct connection but also an indirect connection in which another component exists in between. Furthermore, when a component is said to "include" or "have" another component, unless otherwise specifically stated, this does not exclude the other component, but rather implies that the other component may be included.

[0031] In the present invention, terms such as first, second, etc. are used only for the purpose of distinguishing one component from another component, and do not limit the order or importance between components unless specifically stated otherwise. Therefore, within the scope of the present invention, a first component in one embodiment may be referred to as a second component in another embodiment, and similarly, a second component in one embodiment may be referred to as a first component in another embodiment.

[0032] In the present invention, distinct components are used to clearly illustrate their respective characteristics, and do not necessarily imply that the components are separated. In other words, multiple components may be integrated into a single hardware or software unit, or a single component may be distributed into multiple hardware or software units. Therefore, even if not specifically mentioned, such integrated or distributed embodiments are also included within the scope of the present invention.

[0033] In the present invention, the components described in various embodiments are not necessarily essential components, and some may be optional components. Therefore, embodiments comprising a subset of the components described in one embodiment are also within the scope of the present invention. Furthermore, embodiments that include other components in addition to the components described in various embodiments are also within the scope of the present invention.

[0034] Hereinafter, embodiments of the present invention will be described with reference to the attached drawings.

[0035]

[0036] FIG. 1 is a drawing for explaining a music playback method using blocks according to one embodiment of the present invention.

[0037] According to one embodiment of the present invention, at least one block can be captured via a receiving device mounted on a music playback device (S101). For example, a music playback device refers to a device that can execute an application for playing music by mounting a receiving device. More specifically, it may refer to an OS-based device. Examples include, but are not limited to, an iPad, a smartphone, or a personal computer (PC).

[0038] In addition, the receiving device may be a type of photographing device. That is, the music playback device according to one embodiment of the present invention can photograph at least one block through the mounted photographing device and play music using the blocks included in the photographed scene. In this case, the photographing device may mean a camera, for example, but is not necessarily limited thereto. The photographing device according to one embodiment of the present invention can photograph a wider variety of blocks through functions such as zooming in and zooming out.

[0039] According to one embodiment of the present invention, a music playback device may recognize at least one photographed block using a trained deep learning neural network model (S102). In this case, the deep learning neural network model may be a convolutional neural network model.

[0040] The above deep learning neural network model may determine that the type of the block is at least one of a start block that plays at least one connected instrument block once, a repeat block that repeatedly plays at least one connected instrument block after music of at least one connected instrument block is played, a level block that specifies a performance level of the instrument block, an instrument block corresponding to each instrument for the music playback, an operation block that adjusts the current level when mounted on a block, a condition block that can set the condition of the corresponding condition block to the instrument block until the next condition block appears, a star storage block that can store instrument blocks to be played in advance, and a star execution block that can execute instrument blocks stored in the star storage block all at once, all of which are interconnected blocks. In this case, each of the instruments may be at least one of drums, brass, piano, bass guitar, and lead guitar, but is not necessarily limited thereto, and may refer to various other instruments.

[0041] For example, the deep learning neural network model can input block images or video data included in a scene captured by a camera, and output recognition information of the block with the highest probability. At this time, the recognition information can mean the x-coordinate, y-coordinate of the location where the image is placed for each block within the camera, and a label number corresponding to the type of the block.

[0042] In addition, the music playback device according to one embodiment of the present invention can determine the arrangement structure of the blocks recognized in step S102 (S103). That is, steps S102 and S103 may refer to steps for determining the arrangement structure by determining the coordinate values ​​(e.g., x-coordinate, y-coordinate) of the positions where at least one recognized block is placed and the type of the block.

[0043] In addition, according to one embodiment of the present invention, music can be played on the music playback device based on the determined arrangement structure in the determined step S103 (S104). At this time, the music playback device can play the music based on all of the plurality of block arrangement structures recognized through the receiving device (e.g., camera). For example, if the arrangement structure of the blocks captured by the photographing device is a plurality of sets, music can be played according to the arrangement structure of the blocks corresponding to the plurality of sets. At this time, the playback order of the music can be determined by the type of the blocks.

[0044] A music playback method performed by a music playback device according to one embodiment of the present invention can capture a scene including at least one block through a receiving device (e.g., a camera) and play music based on the captured image. For example, the captured image can be input into a deep learning model. The input process can also be performed using Python. In addition, when the blocks are recognized by the deep learning model, the block arrangement structure is determined through a specific algorithm, and music can be played from the music playback device based on the determined arrangement structure.

[0045]

[0046] Hereinafter, with reference to FIGS. 2 to 10, a music playback algorithm according to an example and arrangement structure of blocks included in an image received by a receiving device (e.g., a camera) according to one embodiment of the present invention will be described.

[0047] FIG. 2 is a drawing for explaining the types of blocks used in a music reproduction method according to one embodiment of the present invention.

[0048] Referring to Figure 2a, a start block and a repeat block are illustrated as examples of control blocks. Here, the start block may refer to a block that, when executed, executes the blocks connected to the start block for the first time. Furthermore, the repeat block may refer to a block that operates after the start block and infinitely repeats the blocks connected to the repeat block (e.g., instrument blocks). Meanwhile, the repeat block can also operate singly.

[0049] Referring to Fig. 2b, various types of instrument blocks are illustrated. Here, the instrument blocks may include, but are not necessarily limited to, drum blocks, brass blocks, piano blocks, bass guitar blocks, and lead guitar blocks. The instrument blocks may refer to blocks in which each corresponding instrument block is played. Each instrument block may have a structure in which a repeat block, a star block, or another instrument block can be inserted into the upper portion.

[0050] Referring to Figure 2c, a Star Block that can be customized to a user's preference is illustrated. The Star Block is comprised of a Star Storage Block and a Star Execution Block. The Star Storage Block may refer to a block that stores a set of instrument blocks connected to the Star Storage Block. Furthermore, the Star Execution Block may refer to a block that executes a set of instrument blocks stored in the Star Storage Block.

[0051] Referring to Figure 2d, a level block is illustrated. A level block is a block mounted in the middle of an instrument block, and may refer to a block that specifies the level (velocity) of the corresponding instrument. For example, an instrument equipped with a level 2 block may play music with a stronger volume than an instrument equipped with a level 1 block. In this case, an instrument block without a level block mounted may be considered level 0.

[0052] FIG. 3 is a drawing for explaining the types of blocks used in a music reproduction method according to one embodiment of the present invention.

[0053] Referring to Figure 3a, an operation block is illustrated. When mounted on a block, the operation block can adjust the current level. For example, an increase block can increase the current instrument playback level by +1. The maximum possible level increase is 4, but is not necessarily limited thereto.

[0054] Additionally, for the reduction block, the currently set instrument playback level can be reduced by -1. The minimum level that can be reduced may be 0, but is not necessarily limited to this.

[0055] Meanwhile, a maintenance block (regeneration block) may refer to a block that maintains the current level. Furthermore, a reset block may refer to a block that resets the level to the initial setting before the operation block appears.

[0056] Referring to Figure 3b, a conditional block is illustrated that determines whether to play connected blocks based on the tilt of the music playback device. For example, in the case of the left tilt block, which is indicated by a left arrow, blocks connected to the bottom of the left tilt block may be executed when the music playback device (e.g., a smartphone) is tilted to the left.

[0057] Additionally, for the right tilt block with the right arrow shown, when the music player device (e.g., smartphone) is tilted to the right, the blocks connected to the bottom of the right tilt block may be executed. On the other hand, for the other tilt blocks, when the music player device (e.g., smartphone) is tilted in a direction other than the currently existing condition block for a specific direction, the blocks connected to the bottom of the other tilt blocks may be executed.

[0058] FIG. 4 is a drawing for explaining a method of arranging blocks for music playback according to one embodiment of the present invention.

[0059] Referring to Fig. 4a, a start block, an instrument block, and a level block are illustrated. Each block can be fitted according to its shape. For example, referring to Fig. 4a, an instrument block can be fitted beneath the start block, and a level block that sets the level of the instrument can be fitted in the center of the instrument block. However, the attachment locations and configurations of the blocks according to the embodiments of the present invention are exemplary, and attachment in various locations and methods is also possible.

[0060] Referring to FIG. 4b, it is illustrated that new instrument blocks can be inserted into the lower part of the block unit set in FIG. 4a.

[0061] Referring to Figure 4c, it is illustrated that when multiple block layout structure units exist within a photographing device (e.g., a camera) included in a music playback device, music pieces according to each layout structure can all be played. In this case, each block group must be arranged with a certain amount of free space to ensure smooth music playback.

[0062]

[0063] FIGS. 5 to 10 are drawings for explaining connection rules of blocks for music playback according to one embodiment of the present invention.

[0064] Referring to Figure 5a, it is illustrated that all instrument blocks must be connected to a control block (start block, repeat block) to be executed. For example, as illustrated in Figure 5a, a bass guitar (level 0 or level 1) must be connected to a start block to be executed, and cannot be executed by a start block alone or an instrument block alone.

[0065] Referring to Fig. 5b, it is illustrated that when different instruments are sequentially connected, they can be played as a group. For example, as illustrated in Fig. 5b, when multiple instrument blocks are connected as a group within a single starting block, the instruments connected to that starting block can be played simultaneously. In this case, according to one embodiment of the present invention, in an application on a music playback device, characters corresponding to each instrument can be played simultaneously on a single screen.

[0066] Referring to Fig. 6a, when instruments of the same type are connected in succession, the instrument blocks can be played in the order in which they are connected closest to the starting block. For example, in the case of the brass block illustrated in Fig. 6a, brass can be played in the order of Level 1, Level 3, and Level 2.

[0067] Referring to Fig. 6b, if instruments of the same type are separated and attached within a single starting block, the instruments can be played as a group until the same instrument appears. For example, as shown in the first picture of Fig. 6b, a level 1 bass guitar and a level 1 lead guitar connected below the starting block can be played together in one measure, and since a level 2 bass guitar appears again below it, they can be played together with a level 2 lead guitar connected below the level 2 bass guitar in two measures. This also applies when the instrument blocks are configured as shown in the second and third pictures of Fig. 6b. That is, the instruments can be played as a group until the same instrument appears again. On the other hand, in the case shown in the fourth picture of Fig. 6b, a level 2 bass guitar is connected directly below a level 1 bass guitar, and since a lead guitar appears again below the lead guitar connected below it, a performance different from the first, second, and third block configurations of Fig. 6b can be performed. That is, in the fourth picture shown in FIG. 6b, a level 1 bass guitar can be played in the first measure, a level 2 bass guitar and a level 1 lead guitar can be played together in the second measure, and a level 2 lead guitar can be played in the third measure.

[0068] Referring to Fig. 6c, in the case of the first illustrated block structure of Fig. 6c, a level 1 bass guitar and a level 1 lead guitar are positioned in the start block, respectively, so a level 1 bass guitar and a level 1 lead guitar can be played in the first measure.

[0069] In the second illustrated block structure of Fig. 6c, a level 1 bass guitar is played in the first measure, and a bass guitar is positioned right after that. Since the level of this bass guitar is 0, a level 0 bass guitar and a level 1 lead guitar can be played together in the second measure.

[0070] Referring to the block structure illustrated in the third of Fig. 6c, a level 0 lead guitar is inserted under a level 1 bass guitar, and a level 1 lead guitar is inserted under that, so that a level 1 bass guitar and a level 0 lead guitar can be played together in one measure, and a level 1 lead guitar can be played in two measures.

[0071] Referring to Fig. 7a, the execution order of the repeat block is illustrated as the start block followed by the repeat block. In the case of the picture shown on the left side of Fig. 7a, since the level 1 bass guitar is connected under the start block and the level 2 bass guitar is connected under the repeat block, it can be seen that the level 1 bass guitar is played once initially, and then the level 2 bass guitar is played repeatedly.

[0072] Meanwhile, in the case of the picture shown on the right side of Fig. 7a, the level 1 bass guitar is executed once, and according to the operation of the increase block, the bass guitar increases to level 2, level 3, and so on, and can be executed infinitely and repeatedly.

[0073] In the case of Figure 7b, it is illustrated that all instrument blocks must always use a level block when they first appear. For example, if the level block directly connected below the start block is level 1, the performance can still be executed even if there is an instrument block connected to an increment block next to it. Furthermore, even if there is a repeat block connected to an increment block along with a start block connected to a level 1 instrument block, the performance can still be executed because the block execution order is the infinite repeat block following the start block.

[0074] Conversely, if an increase block appears before a level block, the instrument may not be played. In this case, the music playback device may recognize the block layout and output an error message instead of playing the music.

[0075] Referring to FIG. 8, it is illustrated that the operation block can perform an operation (increase, decrease, or maintain) on the current level of the corresponding instrument.

[0076] When an increment block is connected to a repeat block, the current level of the corresponding instrument (e.g. level 1) can be increased by one to cause the performance to proceed.

[0077] When a decrease block is connected to a repeat block, the current level of the corresponding instrument (e.g. level 4) can be decreased by one to allow the performance to proceed.

[0078] When a sustain block is connected to a repeat block, the performance can be performed while maintaining the current level of the instrument (e.g., level 2).

[0079] The reset block can be used to reset the instrument's level to its initial setting. For example, as illustrated in Figure 8, the initial level of the bass guitar connected to the start block is 1. Therefore, the reset block connected to the repeat block resets the bass guitar's level to 1, allowing the instrument to be played.

[0080]

[0081] Referring to Figure 9, it is illustrated that the role of a condition block is to set the condition of the corresponding condition block to the instrument block until the next condition block appears.

[0082] For example, referring to Fig. 9, since a brass block is connected under a right tilt block, when the music player device is tilted to the right, the brass block can be played. In addition, since a bass guitar block is connected under a left tilt block, when the music player device is tilted to the left, the bass guitar block can be played. In addition, since a lead guitar block is connected under an else condition block, when the music player device is not tilted, the lead guitar block can be played. In this case, there is no need for multiple condition blocks, and operation can be possible even if only one is connected.

[0083]

[0084] Referring to FIG. 10, the roles and operation methods of the star storage block and the star execution block are illustrated.

[0085] When the instrument blocks to be played are inserted into the star storage block and stored in advance, the instrument blocks stored in the star storage block can be executed all at once through the star execution block. For example, referring to Fig. 10, the drum block, brass block, and piano block of level 1 can be inserted into the star storage block and stored, respectively. At this time, when the start block and the sustain block are inserted into the star execution block, the drum block, brass block, and piano blocks of level 1 that were previously stored in the star storage block can be executed.

[0086] Meanwhile, by inserting an operation block into a star execution block, the function of simultaneously controlling the operations of instruments stored in a star storage block can be performed. For example, if the instrument blocks previously stored in a star storage block are a drum block, a brass block, and a piano block of level 1, inserting an increase block into the star execution block can play the drum block, brass block, and piano block of level 2, respectively.

[0087] According to one embodiment of the present invention, music playback may be performed only when there is no more than one start block, repeat block, and star storage block within a scene recognized by a receiving device (e.g., a camera). On the other hand, if there are two or more blocks of the same type within the scene recognized by the receiving device (e.g., a camera), the music playback device according to one embodiment of the present invention may output an error message and not play music. For example, if there are two or more repeat blocks within the scene recognized by the receiving device (e.g., a camera), the music may not be played.

[0088] According to one embodiment of the present invention, when there is one or less start block, one or less repeat block, and one or less star storage block in a shooting scene recognized by a receiving device (e.g., a camera), each character corresponding to the recognized instrument block can perform a performance through an application running on a music playback device.

[0089]

[0090] FIG. 11 is a drawing for explaining the configuration of a music playback device utilizing blocks according to one embodiment of the present invention.

[0091] The music playback device (1100) illustrated in FIG. 11 may refer to hardware that performs the music playback method illustrated in FIG. 1. The description of the music playback method described above may also be applied to the description of the music playback device (1100) illustrated in FIG. 11.

[0092] The music playback device (1100) may include a receiving device (1110) capable of recognizing and capturing at least one block outside the music playback device, a memory (1120) capable of storing at least one block input through the receiving device, and at least one processor (1130), and may further include other elements not shown in the drawing without limitation. The receiving device may mean, for example, a photographing device.

[0093] The processor (1130) can perform a music reproduction method using the above-described blocks, and for example, the processor (1130) can recognize through the receiving device (1110) mounted on the music reproduction device (1130) using a pre-learned deep learning neural network model, and determine the arrangement structure of at least one block. In addition, the processor (1130) can reproduce the music on the music reproduction device based on the determined arrangement structure.

[0094] In addition, the above processor (1130) can perform at least one operation for performing a music reproduction method using blocks according to the present invention.

[0095] While the exemplary methods of the present invention are presented as a series of operations for clarity of description, this is not intended to limit the order in which the steps are performed, and individual steps may be performed simultaneously or in different orders, if desired. To implement a method according to the present invention, additional steps may be included in addition to the exemplified steps, some steps may be excluded and the remaining steps may be included, or some steps may be excluded and additional steps may be included.

[0096] The various embodiments of the present invention are not intended to list all possible combinations, but rather to illustrate representative aspects of the present invention, and the matters described in the various embodiments may be applied independently or in combinations of two or more.

[0097] In addition, various embodiments of the present invention may be implemented by hardware, firmware, software, or a combination thereof. In the case of hardware implementation, it may be implemented by one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), general processors, controllers, microcontrollers, microprocessors, etc.

[0098] The scope of the present invention includes software or machine-executable instructions (e.g., operating systems, applications, firmware, programs, etc.) that cause operations according to the methods of various embodiments to be executed on a device or a computer, and a non-transitory computer-readable medium having such software or instructions stored thereon and executable on the device or computer.

Claims

1. A music playback method using at least one block performed by a music playback device, A step of capturing an image including at least one block through a receiving device mounted on the music playback device; A step of recognizing at least one block photographed using a learned deep learning neural network model; A step of determining the arrangement structure of the above recognized block; and A step of playing the music on the music playback device based on the determined arrangement structure, How to play music using blocks.

2. In paragraph 1, A method of playing music using blocks, wherein the receiving device is a photographing device.

3. In paragraph 1, The above deep learning neural network model is a convolutional neural network model, a method for playing music using blocks.

4. In paragraph 1, The step of judging the above layout structure is A method for playing music using blocks, which determines the coordinate values ​​of the position where at least one block recognized above is placed and the type of block.

5. In paragraph 1, The above-mentioned deep learning neural network model is, The layout structure of the above blocks, A start block that plays at least one connected instrument block once, After the music of at least one instrument block connected to the above start block is played, a repeat block that repeatedly plays at least one connected instrument block; A level block that specifies the playing level of the above instrument block, and It is determined that at least one of the instrument blocks corresponding to each instrument is a layout structure interconnected for the above music playback. How to play music using blocks.

6. In paragraph 5, A music reproduction method using blocks, wherein the music reproduction device reproduces the music based on all of the plurality of arrangement structures photographed through the receiving device.

7. In a music playback device using blocks, The above device, A receiving device for capturing an image containing at least one block; memory; and Contains at least one processor, The above processor, Recognize at least one block photographed through the receiving device using the learned deep learning neural network model, Determine the layout structure of the above blocks, Playing the music based on the above-determined arrangement structure, A music playback device using blocks.

8. A non-transitory computer-readable recording medium storing a program for implementing a music playback method using the blocks described in any one of paragraphs 1 to 6.

Images