Music generation method and device based on emotion curve

By obtaining and analyzing the emotion curve, generating emotional dimension vectors, and matching music segment libraries, the problem that the existing technology cannot adjust the intensity of emotions is solved, and more refined and flexible music generation is achieved, suitable for movie soundtracks and other needs.

CN119007694AInactive Publication Date: 2024-11-22BEIJING INSTITUTE FOR GENERAL ARTIFICIAL INTELLIGENCE
View PDF 9 Cites 0 Cited by

Patent Information

Application Number
CN202411480871.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-23
Publication Date
2024-11-22
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing music generation methods cannot generate music that can regulate the intensity of emotions, and cannot meet actual needs such as movie soundtracks.

Method used

By obtaining the emotion curve, counting emotional changes, generating emotional dimension vectors that represent the intensity of emotions, and performing vector matching in the trained music clip library, selecting the corresponding music clip to generate music.

Benefits of technology

The fine adjustment of the degree of emotional intensity is achieved, and more advanced and flexible music generation methods are provided, making up for the problems of music discontinuity and inconsistent styles in the prior art.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119007694A_ABST
    Figure CN119007694A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides a music generation method and device based on an emotion curve. The method comprises the following steps: acquiring an emotion curve; in a given time window, counting the change of the emotion curve, and generating an emotion dimension vector representing the emotion intense degree; and performing vector matching in a trained music clip library model based on the emotion dimension vector, and selecting a corresponding music clip so as to generate music. According to the music generation method and device based on the emotion curve disclosed by the embodiment of the invention, more and more accurate emotion change information can be collected by adopting the emotion curve, and a more refined creation means is provided. In addition, according to the music generation method, corresponding musical instrument types and sound shapes can be generated through the input mood intense degree and music breeze. The sound and shape generation makes up for the defects of discontinuous music and non-uniform styles in a note generation algorithm, and is finer and more flexible compared with a generation algorithm of a whole piece of music.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of audio data processing, and in particular to a music generation method and device based on an emotion curve. Background Art

[0002] With the development of audio processing technology and deep learning algorithms, big data model algorithms have also been applied in the field of music creation. For example, by inputting different music attribute information, music with matching styles can be automatically generated, such as automatically generating movie soundtracks, game music, etc.

[0003] Music is of great significance to people's daily life and artistic enjoyment. Music can express the emotions of the composer and also affect the emotions of the listener. An existing music generation method can generate music based on the input emotion label, and the emotion label acts on the entire piece of music. However, this music generation method can only adjust the entire generated music in terms of emotion control, and cannot adjust the intensity of the emotion. The emotion of a piece of music can be switched and fluctuated, so the existing music generation technology cannot meet some practical needs such as movie soundtracks. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a method and device for generating music based on an emotion curve, so as to solve the problem that the existing technical solutions cannot generate music that can adjust the intensity of emotions.

[0005] In order to solve the above technical problems, a specific embodiment of the present invention provides a music generation method based on an emotion curve, including: obtaining an emotion curve; within a given time window, counting the changes in the emotion curve to generate an emotion dimension vector representing the intensity of the emotion; based on the emotion dimension vector, performing vector matching in a trained music clip library model, and selecting the corresponding music clip to generate music.

[0006] In one embodiment, obtaining the emotion curve includes: inputting a completed emotion curve; or capturing the emotion curve drawn in real time through computer vision, including capturing the curve trajectory drawn in real time by human hands.

[0007] In one embodiment, the obtaining of the emotion curve further comprises: obtaining a plurality of emotion curves; each emotion curve corresponds to a different emotion dimension, and a weight of each emotion dimension can be set.

[0008] In one embodiment, within a given time window, the changes in the emotion curve are counted to generate an emotion dimension vector representing the intensity of the emotion, including: according to the set weight of each emotion dimension, within a given time window, a multidimensional emotion dimension vector representing the intensity of emotions of different dimensions is generated; the emotion dimension vector is a feature vector including pleasure, arousal and dominance.

[0009] In one embodiment, training and generating the music clip library model includes: using a VAE model to generate a large number of music clips; screening the generated music clips, combining clips with non-conflicting musical attributes including rhythm, speed, and pitch range to generate a combined clip library; using a Transformer model to train the music clips, and predicting and generating an emotional dimension vector for each music clip.

[0010] In one embodiment, based on the emotion dimension vector, vector matching is performed in a trained music clip library model, and a corresponding music clip is selected, including: matching the emotion dimension vector generated according to the emotion curve with the emotion dimension vector corresponding to each music clip in the music clip library model, and selecting the music clip with the highest emotion dimension vector similarity; the music clip includes a series of sound shape sets, each sound shape includes the time, pitch, loudness, instrument type and music speed of the note.

[0011] In one embodiment, the selecting corresponding music clips and then generating music includes: for the selected multiple music clips, using VAE to interpolate two adjacent music clips to generate a transition segment.

[0012] In order to solve the above technical problems, the specific implementation of the present invention also provides a music generation device based on an emotion curve, including: an emotion curve acquisition module, used to obtain an emotion curve; an emotion vector generation module, used to count the changes in the emotion curve within a given time window, and generate an emotion dimension vector representing the intensity of the emotion; a music generation module, used to perform vector matching in a trained music clip library based on the emotion dimension vector, and select the corresponding music clip to generate music.

[0013] Another aspect of an embodiment of the present invention provides an electronic device, including one or more processors and a storage device, wherein the storage device is used to store executable instructions, and when the executable instructions are executed by the processor, the method of the embodiment of the present invention is implemented.

[0014] Another aspect of an embodiment of the present invention provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the method of the embodiment of the present invention when executed by a processor.

[0015] Another aspect of an embodiment of the present invention provides a computer program, wherein the computer program includes computer executable instructions, and the instructions are used to implement the method of the embodiment of the present invention when being executed.

[0016] The music generation method and device based on emotion curves disclosed in the embodiments of the present invention can use emotion curves to collect more and more accurate information on emotion changes. Compared with other overall generation and emotion-unified music generation methods, it is more advanced, can better understand the creator's intention, and provides a more refined means of creation. In addition, the music generation method of the present invention can also generate corresponding instrument types and sound shapes according to the input emotion intensity and music style. The generation of sound shapes makes up for the shortcomings of discontinuous music and inconsistent styles in the note generation algorithm, and is more refined and flexible than the generation algorithm of the entire piece of music. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings below are part of the specification of the present invention, and illustrate exemplary embodiments of the present invention. Together with the description of the specification, the accompanying drawings are used to explain the principles of the present invention.

[0018] Figure 1 is a processing flow chart of a music generation method based on an emotion curve according to an embodiment of the present invention; Figure 2 A flowchart of the process of generating a music clip model library for training according to an embodiment of the present invention; Figure 3 A schematic diagram of the structure of a music generating device based on an emotion curve according to an embodiment of the present invention; Figure 4 A schematic diagram of capturing a motion trajectory drawn by a human hand according to a specific embodiment of the present invention; Figure 5 For Figure 4 Schematic diagram of instrument selection and sound shape generated by the emotional curve input. DETAILED DESCRIPTION

[0019] In order to make the objectives, technical solutions and advantages of the embodiments of the present invention more clearly understood, the spirit of the contents disclosed by the present invention will be clearly explained with the accompanying drawings and detailed descriptions below. After understanding the embodiments of the contents of the present invention, any technician in the relevant technical field can change and modify the techniques taught by the contents of the present invention without departing from the spirit and scope of the contents of the present invention.

[0020] The exemplary embodiments and descriptions of the present invention are used to explain the present invention, but are not intended to limit the present invention. In addition, elements / components with the same or similar reference numerals used in the drawings and embodiments are used to represent the same or similar parts.

[0021] The terms “first”, “second”, etc. used in this document do not particularly refer to an order or sequence, nor are they used to limit the present invention. They are only used to distinguish elements or operations described with the same technical terms.

[0022] The directional terms used in this document, such as up, down, left, right, front or back, etc., are only used to refer to the directions of the drawings. Therefore, the directional terms used are used to illustrate and not to limit the present invention.

[0023] The words “include,” “including,” “have,” “contain,” etc. used in this document are open-ended terms, meaning including but not limited to.

[0024] As used herein, "and / or" includes any or all combinations of the items described.

[0025] As used herein, “plurality” includes “two” and “more than two”; as used herein, “plurality groups” includes “two groups” and “more than two groups”.

[0026] The terms "substantially" and "approximately" used herein are used to modify any quantity or error that may vary slightly, but these slight changes or errors do not change their essence. Generally speaking, the range of slight changes or errors modified by such terms may be 20% in some embodiments, 10% in some embodiments, 5% in some embodiments, or other values. Those skilled in the art should understand that the aforementioned values ​​can be adjusted according to actual needs and are not limited thereto.

[0027] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0028] In the case of using expressions such as "at least one of A, B, and C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but not be limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or a system having A, B, C, etc.). In the case of using expressions such as "at least one of A, B, or C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, or C" should include but not be limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or a system having A, B, C, etc.). Those skilled in the art should also understand that any transitional conjunctions and / or phrases that substantially represent two or more optional items, whether in the specification, claims, or drawings, should be understood to give the possibility of including one of these items, either of these items, or both of these items. For example, the phrase "A or B" should be understood to include the possibilities of "A" or "B", or "A and B".

[0029] Figure 1 The flowchart of the music generation method based on the emotion curve of the embodiment of the present invention is shown in the figure. As shown in the figure, the method includes: step S101, obtaining the emotion curve; step S102, within a given time window, counting the changes of the emotion curve, and generating an emotion dimension vector representing the intensity of the emotion; step S103, based on the emotion dimension vector, performing vector matching in the trained music clip library, and selecting the corresponding music clip, and then generating music.

[0030] In specific implementation, in step S101, there are multiple ways to obtain the emotion curve. The emotion curve can be a curve that has been drawn, including a curve drawn by a machine or drawn manually with pen and paper, as long as it can describe the characteristics of the change of the intensity of emotion over time. For example, a relatively simple one is an emotion curve with time as the x-axis and the intensity of emotion as the y-axis, and the value is the positive axis of xy.

[0031] Another way to obtain the emotion curve is to capture the emotion curve drawn in real time through computer vision, such as capturing the curve trajectory drawn by human hands in real time, and outputting music in real time while the human hands are drawing.

[0032] Regardless of the acquisition method, since the emotion curve can represent the different degrees of emotion intensity, the height of the emotion curve will continue to change over time.

[0033] In other embodiments, multiple emotion curves may be obtained simultaneously, each emotion curve corresponds to a different emotion dimension, and a weight of each emotion dimension may be set.

[0034] For example, the user can input different emotion dimensions, i.e., emotion types, such as "sad", "magnificent", "happy", etc., and set an emotion curve for each emotion dimension. In this embodiment, the input of each emotion curve can be expressed as: emotion(t) = f_emotion(t)*C_emotion(t) ; If there are multiple emotion dimensions, the input of the emotion curve can be expressed as: ; in, emotion(t) Represents the emotional curve, f_emotion(t) Represents the emotional dimension, w represents the weights of different emotion dimensions, C_emotion(t) A curve showing how emotional intensity changes over time. t Indicates time.

[0035] In specific implementation, in step S102 of this embodiment, according to the set weight of each emotional dimension, a multidimensional emotional dimension vector representing the intensity of emotions of different dimensions is generated within a given time window. In specific implementation, it is shown in the following formula: ; in, E(t) For the generated t Momental emotion vector, t0 Indicates the current time. T represents the selected time window, emotion(t) This is the emotion curve obtained in step S101.

[0036] In step S103 of the present invention, the music clip library model has been learned and trained. In this embodiment, the steps of training and generating the music clip library are as follows: Figure 2 As shown, including: Step S201, using the VAE model to generate a large number of music clip libraries with instruments, sound shapes and chords. The generation of each music clip can be expressed as: ; in, z represents the latent variable, extracted from the music clip by the VAE encoder, represents the mean, 2 Represents variance.

[0037] Step S202, screening the generated music clips to ensure that the clips with non-conflicting music attributes such as rhythm, speed, pitch range, etc. are combined together to generate a combined clip library.

[0038] Step S203, use the Transformer model to train the music clips and predict the emotional dimension vector of each music clip, namely, the pleasure (Valence), arousal (Arousal), and dominance (Dominance) of the emotion. During the model training process, the instrument + MIDI clip is used as input, and the generated VAD emotion vector is expressed as: VAD(pi) = Transformer(pi); in, VAD(pi) represents the emotion vector of the music clip, Transformer() For the trained Transformer model, pi Indicates a piece of music.

[0039] In the embodiment of the present invention, based on the emotion dimension vector, vector matching is performed in the trained music clip library model, and the corresponding music clip is selected, that is, the emotion dimension vector generated according to the emotion curve is matched. E(t) and the emotional dimension vector of each music clip in the music clip library model VAD(pi) Perform matching and select the segment with the most similar sentiment vector: P(match) = min{E(t) - VAD(pi)}; P(match) To find the formula for the corresponding fragment, we minimize t The difference between the emotion vector generated by the emotion curve over time and the emotion vector of the music clip.

[0040] In this embodiment, the music type of the emotion curve can also be set, such as rock music, country music, Chinese folk music, etc. At this time, the matching of the emotion dimension vector and the selection of the music clip are only matched in the model library representing the set music type, which will reduce the complexity of the matching algorithm. For example, if the emotion dimension of the emotion curve only includes "sadness" and "magnificence", the music type can be set to "Chinese folk music", then matching and selection will be performed in the model library of "Chinese folk music", reducing the complexity and calculation amount of the matching algorithm.

[0041] In this embodiment, the music clip includes instrument information and a series of sound shape sets with emotion, style, and mode annotations. The sound shape is composed of a group of notes. The information of the sound shape includes the time, pitch, loudness, instrument type, and music speed of each note. The music generation method of the present invention can generate the corresponding instrument type and sound shape according to the input emotional intensity and style, and then the sound shape will form an audio file and play it. The generation of the sound shape makes up for the shortcomings of discontinuous music and inconsistent style in the note generation algorithm, and is more sophisticated and flexible than the generation algorithm of the entire music.

[0042] In step S103 of the embodiment of the present invention, in order to make the connection between the music clips more natural, VAE is used to interpolate two adjacent music clips to generate transition segments, thereby making the connection between the music clips more natural. The interpolation process can be expressed as: P(transition)=αz1+(1−α)z2,α∈[0,1]; in, z1, z2 The latent variable representing two adjacent music clips in time generates a transition segment through interpolation P(transition), α is the adjustment parameter.

[0043] Based on the same inventive concept, the embodiment of the present invention also discloses a music generating device based on an emotion curve, such as Figure 3 As shown, it includes: an emotion curve acquisition module 301, which is used to obtain the emotion curve; an emotion vector generation module 302, which is used to count the changes in the emotion curve within a given time window and generate an emotion dimension vector that represents the intensity of the emotion; a music generation module 303, which is used to perform vector matching in a trained music clip library based on the emotion dimension vector, and select the corresponding music clip to generate music.

[0044] In this embodiment, the emotion curve acquisition module 301 is used to acquire the emotion curve by: inputting a completed emotion curve; or capturing the emotion curve drawn in real time through computer vision, including capturing the curve trajectory drawn in real time by human hands.

[0045] In this embodiment, the emotion curve acquisition module 301 acquires the emotion curve further comprising: acquiring multiple emotion curves; each emotion curve corresponds to a different emotion dimension, and a weight of each emotion dimension can be set.

[0046] In this embodiment, the emotion vector generation module 302 is used to count the changes in the emotion curve within a given time window and generate an emotion dimension vector that represents the intensity of the emotion, including: based on the set weight of each emotion dimension, generating a multidimensional emotion dimension vector that represents the intensity of emotions of different dimensions within a given time window; the emotion dimension vector is a feature vector including pleasure, arousal and dominance.

[0047] In this embodiment, the music generation module 303 performs vector matching in a trained music clip library model based on the emotion dimension vector and selects a corresponding music clip, including: matching the emotion dimension vector generated according to the emotion curve with the emotion dimension vector corresponding to each music clip in the music clip library model, and selecting the music clip with the highest similarity of the emotion dimension vector; the music clip includes a series of sound shape sets, each sound shape includes the time, pitch, loudness, instrument type and music speed of the note.

[0048] In this embodiment, the music generation module 303 also includes a VAE interpolation unit, which is used to use VAE to interpolate two adjacent music clips for the selected multiple music clips to generate a transition segment.

[0049] The specific implementation of various modules in the music generation device based on emotion curves of this embodiment can be found in the above method embodiments, which will not be repeated here.

[0050] Figure 4 and Figure 5 The method for generating music based on emotion curves of the present invention is used to capture the motion trajectory of hand drawing as the emotion curve, and the method is used to generate music based on emotion curves of the present invention. Figure 4 A schematic diagram of the selection of musical instruments and the sound shape generated by the emotion curve. It can be understood by those skilled in the art that Figure 4 and Figure 5 What is captured is the hand movement trajectory and sound shape generation diagram at a certain moment, but Figure 4 The human hand in the video moves in real time, and the computer also captures the trajectory of the human hand in real time as the input of the ever-changing emotional curve; Figure 5 The types of instruments and the shapes of the sounds shown in the figure will also change as the hand-drawn track moves. Figure 5 At this moment, the sound shape output is the piano and the first group of sound shapes corresponding to the piano [piano, sound shape 1], and the next moment may generate the violin and the third group of sound shapes corresponding to the violin [violin, sound shape 3], or it may be a combination of piano and violin [piano, sound shape 2, violin, sound shape 1], or a combination of more musical instruments and their corresponding sound shapes.

[0051] The music generation method based on the emotion curve implemented by the above embodiment can use the emotion curve to collect more and more accurate emotion change information. It is more advanced than other overall generation and emotion unified music generation methods, can better appreciate the creator's intention, and provides a more refined means of creation. In addition, the music generation method of this embodiment can also generate corresponding instrument types and sound shapes according to the input emotion intensity and style. The generation of sound shapes makes up for the shortcomings of discontinuous music and inconsistent styles in the note generation algorithm, and is more refined and flexible than the generation algorithm of the entire piece of music.

[0052] According to an embodiment of the present invention, the method flow according to an embodiment of the present invention can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. According to an embodiment of the present invention, the electronic device, device, apparatus, module, unit, etc. described above can be implemented by a computer program module.

[0053] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0054] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, apparatus or device.

[0055] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0056] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present invention may be combined and / or combined in various ways, even if such combinations and / or combinations are not explicitly described in the present invention. In particular, the features described in the various embodiments and / or claims of the present invention may be combined and / or combined in various ways without departing from the spirit and teachings of the present invention. All of these combinations and / or combinations fall within the scope of the present invention.

[0057] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination. The scope of the present invention is defined by the attached claims and their equivalents. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A music generation method based on emotion curve, It is characterized in that include: Get the emotion curve; In a given time window, counting the changes in the emotion curve to generate an emotion dimension vector representing the intensity of the emotion; Based on the emotion dimension vector, vector matching is performed in the trained music clip library model, and the corresponding music clip is selected to generate music.

2. The music generation method based on emotion curve according to claim 1, It is characterized in that The obtaining of the emotion curve comprises: Enter a completed emotion curve; or Capturing emotional curves drawn in real time through computer vision, including capturing the trajectory of curves drawn by human hands in real time.

3. The music generation method based on emotion curve according to claim 1, It is characterized in that The obtaining of the emotion curve further comprises: Get multiple emotion curves; Each emotion curve corresponds to a different emotion dimension, and the weight of each emotion dimension can be set.

4. The method for generating music based on emotion curve according to claim 3, It is characterized in that The step of counting the changes of the emotion curve within a given time window to generate an emotion dimension vector representing the intensity of the emotion includes: According to the set weight of each emotional dimension, a multi-dimensional emotional dimension vector representing the intensity of emotions in different dimensions is generated within a given time window; The emotion dimension vector is a feature vector including pleasure, arousal and dominance.

5. The music generation method based on emotion curve according to claim 1, It is characterized in that The training and generation of the music clip library model includes: Generate a large number of music clips using the VAE model; Screening the generated music clips, combining clips with non-conflicting music attributes including rhythm, speed, and pitch range, to generate a combined clip library; The Transformer model is used to train the music clips and predict the emotional dimension vector of each music clip.

6. The music generation method based on emotion curve according to claim 1, It is characterized in that The step of performing vector matching in a trained music clip library model based on the emotion dimension vector and selecting a corresponding music clip comprises: Matching the emotion dimension vector generated according to the emotion curve with the emotion dimension vector corresponding to each music clip in the music clip library model, and selecting the music clip with the highest emotion dimension vector similarity; The music clip includes a series of sound shape sets, each of which includes the time, pitch, loudness, instrument type and music speed of a note.

7. The method for generating music based on emotion curve according to any one of claims 1 to 6, It is characterized in that The step of selecting the corresponding music clip and then generating the music comprises: For the selected multiple music clips, VAE is used to interpolate two adjacent music clips to generate transition segments.

8. A music generation device based on emotion curve, It is characterized in that include: An emotion curve acquisition module, used to acquire the emotion curve; An emotion vector generation module is used to count the changes of the emotion curve within a given time window and generate an emotion dimension vector representing the intensity of the emotion; The music generation module is used to perform vector matching in a trained music clip library based on the emotion dimension vector, select corresponding music clips, and then generate music.

9. The music generating device based on emotion curve according to claim 8, It is characterized in that The emotion curve acquisition module is used to acquire the emotion curve and includes: Enter a completed emotion curve; or Capturing emotional curves drawn in real time through computer vision, including capturing the trajectory of curves drawn by human hands in real time.

10. The music generating device based on emotion curve according to claim 8, It is characterized in that The emotion curve acquisition module acquires the emotion curve further comprising: Get multiple emotion curves; Each emotion curve corresponds to a different emotion dimension, and the weight of each emotion dimension can be set.

11. The music generating device based on emotion curve according to claim 10, It is characterized in that The emotion vector generation module is used to count the changes of the emotion curve within a given time window and generate an emotion dimension vector representing the intensity of emotion, including: According to the set weight of each emotional dimension, a multi-dimensional emotional dimension vector representing the intensity of emotions in different dimensions is generated within a given time window; The emotion dimension vector is a feature vector including pleasure, arousal and dominance.

12. The music generating device based on emotion curve according to claim 8, It is characterized in that The music generation module performs vector matching in the trained music clip library model based on the emotion dimension vector and selects the corresponding music clip, including: Matching the emotion dimension vector generated according to the emotion curve with the emotion dimension vector corresponding to each music clip in the music clip library model, and selecting the music clip with the highest emotion dimension vector similarity; The music clip includes a series of sound shape sets, each of which includes the time, pitch, loudness, instrument type and music speed of a note.

13. The music generating device based on emotion curve according to any one of claims 8 to 12, It is characterized in that The music generation module also includes a VAE interpolation unit, which is used to use VAE to interpolate two adjacent music clips for the selected multiple music clips to generate a transition segment.

14. An electronic device comprising one or more processors and a storage device, It is characterized in that The storage device is used to store executable instructions, and when the executable instructions are executed by the above-mentioned processor, the music generation method based on the emotion curve as described in any one of claims 1-7 is implemented.

15. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, enable the processor to execute the music generation method based on an emotion curve as described in any one of claims 1-7.

16. A computer program, It is characterized in that The computer program includes computer executable instructions, which, when executed, are used to implement the music generation method based on emotion curves as described in any one of claims 1-7.

Citation Information

Patent Citations

  • A method of matching music authoring by gesture trajectory input

    CN111428075A

  • Music generation method and device, electronic equipment and storage medium

    CN111680185A

  • Emotional music generation method based on deep neural network and music element driving

    CN113299255A

  • Emotion curve generation method of virtual object

    CN115392227A

  • Emotion guidance system based on brain wave music generation of LongFormer

    CN115969377A