Method, system and device for generating music melody based on multi-line feature information

Abstract

The present invention discloses a method, system and device for generating a music melody from multi-line feature information. The method of the present invention analyzes the width of each unit length line in a multi-line picture, the height from the center reference line, the color and other information, converts them into the volume, pitch, timbre and other numerical values ​​in the music, and synthesizes a music melody based on them. Through the method of the present invention, audio-visual combination can be applied to the field of children's music education, which is believed to stimulate children's interest in music and painting art, and cultivate musical sense and imagination.

Description

Technical Field

[0001] The present invention relates to the fields of computer image analysis and music, and in particular to a method, system and device for generating music melody based on multi-line feature information. Technical Background

[0002] In recent years, computer-related technologies have developed rapidly, among which the development of computer image analysis technology is particularly prominent. As this technology becomes more and more mature, the application of image analysis is becoming more and more extensive, such as security face recognition, medical image processing, and image retrieval.

[0003] Music is an art form and cultural activity, and its medium is the regularly organized sound waves (a kind of mechanical wave). Its basic elements include strength, tonality, duration, timbre, etc. These basic elements are combined to form the commonly used "formal elements" of music, such as rhythm, melody, harmony, as well as dynamics, speed, mode, form, texture, etc. The formal elements that constitute music are the means of expression of music.

[0004] At present, music art is becoming more and more diverse, but for some people who have never been exposed to professional music theory knowledge, it is very difficult to create a unique piece of music by themselves. Traditional music theory teaching is also boring for children. Although computer image analysis technology is becoming more and more mature, there is no processing method involving the combination of music and images in the existing technology, especially the method of generating music through simple pictures. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention provides a method, system and device for generating music melody from multi-line feature information, which can synthesize a piece of music according to predetermined rules through information in a picture containing multiple colored lines drawn according to predetermined settings.

[0006] To achieve the above object, the technical solution of the present invention is:

[0007] A first aspect of an embodiment of the present invention provides a method for generating a music melody based on multi-line feature information, which specifically includes the following steps:

[0008] (1) Setting the parameters required for converting multi-line images into music;

[0009] (2) generating a drawing tool according to the parameters set in step (1), and drawing a multi-line image by using the tool;

[0010] (3) Obtaining pixel information of the multi-line image, cutting the pixel information, and extracting feature values ​​in the multi-line image;

[0011] (4) Analyze the feature values ​​in the multi-line image to obtain music information;

[0012] (5) Using music information to generate unit music clips, synthesizing the unit music clips to output music melodies.

[0013] Furthermore, the parameters of step (1) include beat length information corresponding to unit length, beat frequency, expected total music duration, volume range, pitch range and RGB values ​​corresponding to musical instruments.

[0014] Furthermore, in step (2), a drawing board and a brush in a painting tool are generated according to the parameters set in step (1), and the pixel value of the drawing board width is calculated by the beat length information corresponding to the unit length, the beat frequency, and the expected total music duration, thereby generating a blank drawing board; a line is drawn using the brush, and the color of the line is the RGB value corresponding to the instrument set in step (1); the thickness of the line drawn by the brush is positively correlated with the duration of mouse pressing.

[0015] Furthermore, the step (3) specifically includes the following sub-steps:

[0016] (3.1) Obtaining pixel information of multi-line images;

[0017] (3.2) Cut the multi-line image into equal lengths horizontally according to unit length, extract the unit image feature value, and obtain the line block information within the unit length.

[0018] Furthermore, the unit image feature value is an average height of each line block, a relative distance between a center point and a tone reference line, and an RGB value.

[0019] Furthermore, the step (4) specifically includes the following sub-steps:

[0020] (4.1) Obtain the average height of each line block, correspond the maximum average height of the unit line block to the highest decibel, the minimum average height to the lowest decibel, and the other average heights to the decibel size according to the linear rule, and convert the height into a volume value;

[0021] (4.2) obtaining the relative distance between each line block and the pitch reference line, taking the horizontal line of the width center of each line block as the reference line, and taking the pitch range corresponding to the reference line as the pitch value;

[0022] (4.3) Obtain the RGB value of each line block, and obtain the timbre value of the music according to the RGB value corresponding to the instrument set in step (1).

[0023] Furthermore, step (5) specifically includes the following sub-steps:

[0024] (5.1) using the volume value obtained in step (4.1), the pitch value obtained in step (4.2), and the timbre value obtained in step (4.3), a music clip of unit length is generated;

[0025] (5.2) Combine all music clips of unit length to synthesize a music melody, and export the combined music file.

[0026] A second aspect of an embodiment of the present invention provides a system for generating music melodies based on multi-line feature information, including a conversion value setting module, a multi-line drawing module, an image feature value extraction module, a feature value analysis conversion module, and a music synthesis module;

[0027] The conversion value setting module is used to set the parameters required for converting the multi-line image into music;

[0028] The multi-line drawing module draws a multi-line image using the parameters set by the conversion value setting module;

[0029] The image feature value extraction module is used to extract feature values ​​in a multi-line image;

[0030] The eigenvalue analysis and conversion module is used to analyze the eigenvalues ​​in the multi-line image to obtain music information;

[0031] The music synthesis module generates unit music segments using music information, and synthesizes the unit music segments to output music melodies;

[0032] A third aspect of an embodiment of the present invention provides a device for generating a music melody based on multi-line feature information, comprising one or more processors for implementing the above-mentioned method for generating a music melody based on multi-line feature information.

[0033] A fourth aspect of an embodiment of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, is used to implement the above-mentioned method for generating a music melody based on multi-line feature information.

[0034] The beneficial effects of the present invention are as follows: the method of the present invention uses computer image analysis technology to process a picture containing multiple lines, extracts characteristic information of each line, including information characteristic values ​​such as width, height from a horizontal line, and color, and converts these information characteristic values ​​into relevant elements of music using a set of established rules to synthesize a piece of music. The method of the present invention further expands and refines the information range of the lines, and can also make the generated music more diversified. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a module architecture diagram of the system of the present invention;

[0036] Figure 2is a flow chart of the method of the present invention;

[0037] Figure 3 Explain use cases for multi-line drawing of the present invention;

[0038] Figure 4 A line drawing drawn for Example 1 of the present invention;

[0039] Figure 5 The spectrum diagram derived from Example 1 of the present invention;

[0040] Figure 6 Schematic diagram of the device of the present invention. Specific implementation methods

[0041] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0042] The terms used in the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The singular forms "a", "the" and "the" used in the present invention and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0043] The following is a detailed description of a method, system and device for generating music melody based on multi-line feature information proposed by the present invention in conjunction with the accompanying drawings. In the absence of conflict, the features of the following embodiments and implementations can be combined with each other.

[0044] like Figure 1 As shown, the present invention proposes a system for generating music melody based on multi-line feature information, including a conversion value setting module, a multi-line drawing module, an image feature value extraction module, a feature value analysis conversion module and a music synthesis module.

[0045] The conversion value setting module is used to set the parameters required for converting a multi-line image into music; the parameters include unit length (the unit length is custom set, such as taking a width of ten pixels as a unit length), corresponding beat length information, beat frequency (number of beats per minute), expected total music duration, volume range, pitch range and RGB values ​​corresponding to various musical instruments. The conversion value setting module has built-in several mainstream musical instruments, such as piano, violin, cello, guitar, drum set, etc. The RGB values ​​corresponding to the musical instruments are custom set so that lines with corresponding RGB values ​​can be generated in the multi-line drawing module, and the lines correspond to the timbre of the mainstream musical instruments. For example, if the piano is set to red (#FF0000) and the violin is set to yellow (#FFFF00), the multi-line drawing module will generate two brushes, red and yellow.

[0046] The multi-line drawing module uses the parameters set by the conversion value setting module to draw a multi-line image; the multi-line drawing module in the embodiment of the present invention is a drawing tool similar to the built-in simple Windows system, but its drawing board and brush are generated according to the information in the conversion value setting module, and the pixel value of the drawing board width is calculated by the unit length corresponding to the beat length information, the beat frequency, and the expected total music duration, and a blank drawing board with a fixed width of pixels is derived. The corresponding brush color that can be used is set according to the RGB value corresponding to the musical instrument, and the thickness of the brush drawing is related to the mouse pressing time.

[0047] The image feature value extraction module uses the drawing tool obtained in the multi-line drawing module to draw a multi-line image, and passes the exported image to this module. This module will horizontally divide the image according to the unit length information set in module one, and obtain the feature values ​​in each unit image, including the average height of each line block, the relative distance between the center point and the tone baseline (i.e., the horizontal center line of the picture), and the RGB value. These three information are encapsulated into a unit information and passed into the feature value analysis conversion module.

[0048] The eigenvalue analysis and conversion module analyzes the eigenvalues ​​in the multi-line image, and converts the three types of information in each unit information into corresponding music information - volume value, pitch value, and timbre value according to the rules set in the conversion value setting module.

[0049] The music synthesis module generates unit music clips using the music information converted in the eigenvalue analysis conversion module, and then combines all the music clips into a complete music melody and exports it.

[0050] Based on the introduction of the above modules, Figure 2And some assumed setting information data samples are used to further describe in detail the implementation steps of the method for generating music melody based on multi-line feature information proposed in the present invention.

[0051] The implementation steps of the method of the present invention include the following steps.

[0052] (1) Using the conversion value setting module to set the parameters required for converting the multi-line image into music;

[0053] For example, the width of 5 pixels arranged horizontally is a unit length, and a unit length is set to 1 / 4 of the corresponding music beat, and 80 beats per minute are set. According to this standard, if a one-minute piece of music is required, a 1600-pixel wide drawing board will be generated; customize the RGB values ​​corresponding to different instruments, such as red (#FF0000) for piano and yellow (#FFFF00) for violin; set the music pitch range, and the vertical coordinate position of the drawing board will correspond to different tones; set the volume range, the thickest value of the line drawn by the brush will correspond to the highest volume, and the thinnest value will correspond to the lowest volume.

[0054] (2) Draw a multi-line image using a multi-line drawing module. The multi-line drawing module generates a simple drawing tool according to the parameters set in step (1), and draws a multi-line image using the tool.

[0055] The drawing board and brush in the painting tool are generated according to the information in the conversion value setting module. The pixel value of the drawing board width is calculated by the unit length corresponding to the beat length information, the beat frequency, and the expected total music duration, and a blank drawing board with a fixed width of pixels is exported. For example: in the conversion value setting module, the unit length is set to 5 pixels, one unit length is set to 1 / 4 beat of the corresponding music beat, 80 beats per minute are set, and the expected total music duration is 1 minute, then a drawing board with a width of 5*4*80=1600 pixels will be generated, and the height is set to 800 pixels according to the aspect ratio of 2:1. According to the RGB value setting corresponding to the instrument, the corresponding brush color can be used to draw lines, and the thickness of the brush drawing line is positively correlated with the mouse press duration.

[0056] (3) Extracting feature values ​​from the multi-line image using an image feature value extraction module; specifically, the following sub-steps are included:

[0057] (3.1) Obtain pixel information of multi-line images.

[0058] (3.2) The multi-line image is cut horizontally into equal lengths according to the unit length, that is, the previously acquired image information is cut into equal lengths according to the unit information set in step (1), and then the unit image feature values ​​are extracted - the average height of each line block, the relative distance between the center point and the tone baseline, and the RGB value, and these three information are encapsulated into a unit information to obtain the line block information within the unit length.

[0059] (4) Analyze the line block information within the unit length using the eigenvalue analysis conversion module, and perform analysis according to the following steps based on the unit image information obtained in step (3):

[0060] (4.1) Obtain the average height of each line block and convert it into volume value data according to established rules; the established rules are specifically based on the previously set volume range, for example, the highest volume is 60 decibels and the lowest volume is 30 decibels. The maximum average height of the unit line block corresponds to the highest decibel, the minimum average height corresponds to the lowest decibel, and other average heights correspond to decibel sizes according to linear rules.

[0061] (4.2) Obtain the relative distance between each line block and the pitch reference line. In the embodiment of the present invention, the reference line is selected by default as the horizontal line of the width center of an image, and converted into pitch value data according to a predetermined rule; the predetermined rule is according to the previously set pitch interval, for example, set to C major (1 2 3 4 5 6 7 ), the highest pitch corresponds to the high pitch The lowest tone corresponds to the bass 1, and the position above the center horizontal line is divided into 4 equal parts, corresponding to 5, 6, 7, The position below the center horizontal line corresponds to 1, 2, 3, and 4. The pitch value corresponding to the horizontal interval in which the center height of the line block falls will be converted.

[0062] (4.3) Obtain the RGB value of each line block, and generate the timbre value of the music according to the RGB value corresponding to the instrument set in step (1). The colors corresponding to different instruments can be set before conversion, which can be used as the basis for conversion.

[0063] (5) Using the music synthesis module to generate a unit of music fragment according to the music information analyzed in step (5), and synthesizing and outputting the music melody; specifically, the following sub-steps are included:

[0064] (5.1) Synthesize the sound in a set unit beat into a segment, obtain a unit of music data, and generate a unit of music segment. Specifically, use the music data of a unit length, such as 1 / 4 beat, obtained in step (4), that is, the volume value obtained in step (4.1), the pitch value obtained in step (4.2), and the timbre value obtained in step (4.3). Based on these three values, a music segment of unit length (1 / 4 beat) can be generated.

[0065] (5.2) Combine all the segments to synthesize music. Combine the music segments in step (5.1) to synthesize a music melody. Output the combined music file.

[0066] Example 1

[0067] A method for generating a music melody based on multi-line feature information provided by an embodiment of the present invention comprises the following steps:

[0068] (1) Set the parameters required for converting the multi-line image into music: set the unit length to 100 pixels; the beat corresponding to one unit length is 1 beat / unit length; the beat frequency rhythm is 80 beats / minute; the expected total music duration is 24 seconds; the volume range is 30 dB to 60 dB; the pitch range is C major; and the RGB value corresponding to the piano tone is #FFFFFF.

[0069] (2) Generate a drawing board with a width of 800 pixels and a height of 400 pixels according to the parameters set in step (1) with an aspect ratio of 2:1, and a black brush. The time signature is 4 / 4. Draw a multi-line image, such as Figure 4 shown.

[0070] (3) Extract feature values ​​from multi-line images.

[0071] (4) Analyze the line block information within unit length.

[0072] (5) Generate a unit of music fragment according to the music information analyzed in step (5), and synthesize and output the music melody; the music score is as follows Figure 5 shown.

[0073] Corresponding to the aforementioned embodiment of the method for generating a music melody based on multi-line feature information, the present invention also provides an embodiment of a device for generating a music melody based on multi-line feature information.

[0074] See also Figure 6 An embodiment of the present invention provides a device for generating a music melody based on multi-line feature information, comprising one or more processors for implementing the method for generating a music melody based on multi-line feature information in the above embodiment.

[0075] The embodiment of the device for generating music melody based on multi-line feature information of the present invention can be applied to any device with data processing capability, and the device with data processing capability can be a device or apparatus such as a computer. The device embodiment can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, the processor of any device with data processing capability reads the corresponding computer program instructions in the non-volatile memory into the memory and runs them. From the hardware level, if Figure 6 As shown in the figure, it is a hardware structure diagram of any device with data processing capability where the device for generating music melody based on multi-line feature information of the present invention is located. Figure 6 In addition to the processor, memory, network interface, and non-volatile memory shown, any device with data processing capabilities in which the apparatus in the embodiments is located may also include other hardware, generally based on the actual functions of the device with data processing capabilities, which will not be described in detail.

[0076] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0077] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the present invention. Ordinary technicians in this field can understand and implement it without paying creative work.

[0078] An embodiment of the present invention further provides a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, the method for generating a music melody based on multi-line feature information in the above embodiment is implemented.

[0079] The computer-readable storage medium may be an internal storage unit of any device with data processing capability described in any of the aforementioned embodiments, such as a hard disk or a memory. The computer-readable storage medium may also be any device with data processing capability, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit of any device with data processing capability and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capability, and may also be used to temporarily store data that has been output or is to be output.

[0080] In summary, the method of the present invention can be well applied to the fields of children's music teaching interaction, etc., to stimulate children's interest in music and painting art, and cultivate musicality and imagination. At the same time, if we follow the rules of this method and operate in reverse, and use music to generate multi-line images, the generated line zigzag image and music can be combined to achieve audition and listening, which can also better help us deepen our impression and understanding of music.

[0081] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for generating a music melody based on multi-line feature information, characterized in that: The specific steps include: (1) Setting parameters required for converting a multi-line image into music; the parameters include information on the beat length corresponding to a unit length, a beat frequency, an expected total music duration, a volume range, a pitch range, and RGB values ​​corresponding to musical instruments; (2) generating a drawing tool according to the parameters set in step (1), and drawing a multi-line image by using the tool; (3) Obtaining pixel information of the multi-line image, cutting the pixel information, and extracting feature values ​​in the multi-line image; (4) Analyze the feature values ​​in the multi-line image to obtain music information; Wherein, the step (4) specifically includes the following sub-steps: (4.1) Obtain the average height of each line block, correspond the maximum average height of the unit line block to the highest decibel, the minimum average height to the lowest decibel, and the other average heights to the decibel size according to the linear rule, and convert the height into volume value; (4.2) Obtain the relative distance between each line block and the pitch baseline, take the center horizontal line of the width of each line block as the baseline, and take the pitch range corresponding to the baseline as the pitch value; (4.3) Obtain the RGB value of each line block, and obtain the timbre value of the music according to the RGB value corresponding to the instrument set in step (1); (5) Use music information to generate unit music clips, and synthesize the unit music clips to output music melodies.

2. The method for generating a music melody based on multi-line feature information according to claim 1, characterized in that: In the step (2), a drawing board and a brush in a painting tool are generated according to the parameters set in the step (1); the pixel value of the drawing board width is calculated by the beat length information corresponding to the unit length, the beat frequency, and the expected total music duration, and a blank drawing board is generated; a line is drawn using the brush, and the color of the line is the RGB value corresponding to the instrument set in the step (1); the thickness of the line drawn by the brush is positively correlated with the mouse pressing time.

3. The method for generating a music melody based on multi-line feature information according to claim 1, characterized in that: The step (3) specifically includes the following sub-steps: (3.1) Obtain pixel information of multi-line images; (3.2) Cut the multi-line image into equal-length horizontal sections according to unit length, extract the unit image feature values, and obtain the line block information within the unit length.

4. The method for generating a music melody based on multi-line feature information according to claim 3, characterized in that: The unit image feature values ​​are the average height of each line block, the relative distance between the center point and the tone reference line, and the RGB value.

5. The method for generating a music melody based on multi-line feature information according to claim 1, characterized in that: Step (5) specifically includes the following sub-steps: (5.1) Using the volume value obtained in step (4.1), the pitch value obtained in step (4.2), and the timbre value obtained in step (4.3), generate a music clip of unit length; (5.2) Combine all music clips of unit length to synthesize a music melody, and export the combined music file.

6. A system for generating music melody based on multi-line feature information, characterized in that: A method for generating a music melody based on multi-line feature information according to any one of claims 1 to 5, comprising a conversion value setting module, a multi-line drawing module, an image feature value extraction module, a feature value analysis conversion module and a music synthesis module; The conversion value setting module is used to set the parameters required for converting the multi-line image into music; The multi-line drawing module draws a multi-line image using the parameters set by the conversion value setting module; The image feature value extraction module is used to extract feature values ​​in a multi-line image; The eigenvalue analysis and conversion module is used to analyze the eigenvalues ​​in the multi-line image to obtain music information; The music synthesis module generates unit music segments using music information, and synthesizes the unit music segments to output music melodies.

7. A device for generating music melody based on multi-line feature information, characterized in that: It comprises one or more processors for implementing the method for generating music melody based on multi-line feature information as described in any one of claims 1-5.

8. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, it is used to implement the method for generating a music melody based on multi-line feature information as described in any one of claims 1 to 5.

Images