System

Abstract

An object of the system according to the embodiment is to enable a beginner of composition to easily generate and learn music.SOLUTION: A system includes a condition designation unit, a music generation unit, an annotation generation unit, a recommendation unit, and a complement unit. The condition designation unit receives a parameter designated by a user. The musical piece generation part generates a musical piece on the basis of the parameter received by the condition designation part. An annotation generation part generates an annotation sentence of a composition point of the music generated by the music generation part. The recommendation unit recommends an actual music piece having an atmosphere close to that of the music piece generated by the music piece generation unit. The complementing part complements the music based on the user's own melody.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional techniques do not provide sufficient support for beginner composers to easily create and learn songs, and there is room for improvement.

[0005] The system according to the embodiment aims to enable beginners in composition to easily create and learn songs. [Means for solving the problem]

[0006] The system according to the embodiment includes a condition specification unit, a music generation unit, an annotation generation unit, a recommendation unit, and a completion unit. The condition specification unit receives parameters specified by a user. The music generation unit generates a music based on the parameters received by the condition specification unit. The annotation generation unit generates annotations of compositional points of the music generated by the music generation unit. The recommendation unit recommends actual music that has a similar atmosphere to the music generated by the music generation unit. The completion unit completes the music based on a melody created by the user. [Effects of the Invention]

[0007] The system according to the embodiment allows beginners to compose music to easily create and learn songs. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10]1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE INVENTION

[0009] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), and Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (for example, a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., audio and / or text). Display 40A displays visible information such as text and images in accordance with instructions from processor 46. Speaker 40B outputs audio in accordance with instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device owned by a user (e.g., a mobile phone, a robot, a home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) The composition support system according to the embodiment of the present invention automatically generates compositions based on parameters specified by the user, generates annotations for compositional points, recommends actual songs, and complements compositions based on the user's own melodies. This allows even beginners to easily enjoy composing.

[0029] A composition assistance system according to an embodiment includes a condition specification unit, a music generation unit, an annotation generation unit, a recommendation unit, and a completion unit. The condition specification unit receives parameters specified by a user. For example, the user can specify parameters such as major or minor key, time signature, tempo, composition format, music length, and a specific style. The music generation unit generates a music piece based on the parameters received by the condition specification unit. For example, when generating a Bach-style music piece, the generation AI uses a model that has learned Bach's compositional style to generate a Bach-style harmonic progression and melody line. The annotation generation unit generates annotations for the compositional points of the music piece generated by the music generation unit. For example, the generation AI generates annotations that explain the characteristics of the main melody, key points in the harmonic progression, and the relationship with the secondary melody. The recommendation unit recommends actual music pieces that have a similar atmosphere to the music piece generated by the music generation unit. For example, if the generated music piece is Bach-style, the recommendation unit recommends an actual Bach piece. The completion unit completes a song based on a user's original melody. For example, based on a melody created by the user, the generation AI adds a harmonic progression, a counter melody, and a bass part to complete the song. This allows the composition support system according to the embodiment to make it easy for even beginners to enjoy composing. For example, if a user wants to create a Bach-style 4 / 4 song, the user simply specifies the conditions, and the generation AI automatically generates a Bach-style song and provides annotations on compositional points and actual songs for reference. Users can also complete a song based on a melody they created themselves.

[0030] The condition specification unit can automatically present reference materials for the generation AI to draw inspiration from works by past composers based on parameters specified by the user. For example, the condition specification unit can cause the generation AI to automatically search for works by Bach or Beethoven based on user-specified conditions and present them as reference materials. For example, if a user wants to create a Bach-style piece, the condition specification unit can list Bach's representative works. The condition specification unit can also cause the generation AI to analyze the works of a specific composer based on user-specified parameters and provide the analysis results as reference materials. For example, the condition specification unit can analyze Beethoven's sonata-form works and show their structure. The condition specification unit can also cause the generation AI to provide reference materials based on music theory based on user-specified conditions. For example, if a user wants to create a 4 / 4 Allegro piece, the condition specification unit can present information on 4 / 4 rhythm patterns and Allegro tempos. This allows the user to draw inspiration from the works of past composers.

[0031] The condition specification unit enables the generation AI to create hybrid compositions that combine multiple different compositional styles based on user-specified parameters. For example, the condition specification unit enables the generation AI to generate a piece that combines Bach-style and Beethoven-style elements based on user-specified conditions. For example, a Beethoven-style melody line is added to a Bach-style harmonic progression. The condition specification unit also enables the generation AI to generate a hybrid piece that combines classical and jazz elements based on user-specified conditions. For example, jazz improvisation is incorporated into a classical form. The condition specification unit also enables the generation AI to generate a piece that combines compositional styles from different eras based on user-specified conditions. For example, a Romantic melody is added to a Baroque-style harmonic progression. This allows the user to create new pieces that combine multiple compositional styles.

[0032] The condition specification unit provides a visual selection interface for the user when specifying composition conditions, enabling intuitive parameter setting. The condition specification unit, for example, provides a visual selection interface for the user, allowing parameters such as major or minor key, time signature, and tempo to be set by drag and drop. For example, sliders or drop-down menus are used. The condition specification unit also provides a visual selection interface for the user, allowing the user to select the composition form and length of the piece using icons or graphics. For example, by clicking an icon for sonata form or rondo form. The condition specification unit also provides a visual selection interface for the user, allowing the user to select a certain style using an image or sample sound source. For example, by clicking an image of Bach or Beethoven. This allows the user to visually set composition conditions.

[0033] The condition specification unit allows the generation AI to propose different instrumental arrangements and arrangements based on parameters specified by the user. For example, the condition specification unit allows the generation AI to propose different instrumental arrangements, such as piano solo, string quartet, or orchestra, based on conditions specified by the user. For example, the condition specification unit suggests playing a Bach-style piece with a string quartet. The condition specification unit also allows the generation AI to propose different arrangements based on conditions specified by the user. For example, the condition specification unit suggests arranging the same melody in a jazz, classical, or pop style. The condition specification unit also allows the generation AI to propose different instrument combinations based on conditions specified by the user. For example, the condition specification unit suggests combinations such as piano and flute, guitar and violin, etc. This allows the user to propose different instrumental arrangements and arrangements.

[0034] The music generation unit can incorporate not only the style of a specified composer, but also the styles of other composers who influenced that composer. For example, when the generation AI generates a Bach-style piece, the music generation unit also incorporates the styles of Vivaldi and Telemann, who influenced Bach. For example, Vivaldi-style rhythms are added to Bach-style harmonic progressions. When the generation AI generates a Beethoven-style piece, the music generation unit also incorporates the styles of Mozart and Haydn, who influenced Beethoven. For example, Mozart-style ornaments are added to Beethoven-style melody lines. When the generation AI generates a Chopin-style piece, the music generation unit also incorporates the styles of Bach and Mozart, who influenced Chopin. For example, Bach-style counterpoint is added to a Chopin-style piano piece. This makes it possible to generate music that incorporates not only the style of a composer specified by the user, but also the styles of other composers who influenced that composer.

[0035] The music generation unit can generate new music by combining styles from different periods of a specified composer. For example, when the generation AI generates a Bach-style piece, the music generation unit combines the styles of Bach's early, middle, and late works. For example, it combines an early fugue form with a late cantata form. When the generation AI generates a Beethoven-style piece, the music generation unit combines the styles of Beethoven's early, middle, and late works. For example, it combines an early sonata form with a late symphony form. When the generation AI generates a Mozart-style piece, the music generation unit combines the styles of Mozart's early, middle, and late works. For example, it combines an early opera form with a late piano concerto form. This allows the generation of new music by combining styles from different periods of a composer specified by the user.

[0036] The music generation unit can blend musical styles from different cultural spheres in addition to the style of a specified composer. For example, when the generation AI generates a Bach-style piece, the music generation unit blends African rhythms into the Bach style. For example, African rhythm percussion is added to a Bach-style harmonic progression. When the generation AI generates a Beethoven-style piece, the music generation unit blends Asian musical elements into the Beethoven style. For example, traditional Chinese instruments are added to a Beethoven-style melody line. When the generation AI generates a Chopin-style piece, the music generation unit blends Latin music elements into the Chopin style. For example, samba rhythms are added to a Chopin-style piano piece. This makes it possible to generate a piece that blends musical styles from different cultural spheres in addition to the style of a composer specified by the user.

[0037] The music generation unit can arrange the style of a specified composer in a modern style and incorporate modern elements. For example, when the generation AI generates a Bach-style piece, the music generation unit adds elements of modern electronic music to the Bach style. For example, it adds synthesizer tones to a Bach-style harmonic progression. Furthermore, when the generation AI generates a Beethoven-style piece, the music generation unit adds elements of modern pop music to the Beethoven style. For example, it adds pop rhythms to a Beethoven-style melody line. Furthermore, when the generation AI generates a Chopin-style piece, the music generation unit adds elements of modern jazz to the Chopin style. For example, it adds jazz improvisation to a Chopin-style piano piece. In this way, the style of a composer specified by the user can be arranged in a modern style and a piece incorporating modern elements can be generated.

[0038] The annotation generation unit can also explain specific performance techniques and methods of expression in addition to annotations of compositional points. For example, the generation AI may explain performance techniques for the main melody in addition to annotations of compositional points. For example, it may detail the phrasing and articulation methods of the main melody. Furthermore, the annotation generation unit may explain methods for expressing harmonic progression in addition to annotations of compositional points. For example, it may detail the dynamics and tempo changes of the harmonic progression. Furthermore, the annotation generation unit may explain the relationship with the secondary melody in addition to annotations of compositional points. For example, it may detail how the secondary melody complements the main melody. This allows the user to understand specific performance techniques and methods of expression.

[0039] The annotation generation unit can provide annotations for compositional points from multiple perspectives. For example, the generation AI provides annotations for compositional points from the composer's perspective. For example, it explains the composer's intention in choosing a particular harmonic progression. The annotation generation unit also provides annotations for compositional points from the performer's perspective. For example, it explains how a performer expresses the main melody. The annotation generation unit also provides annotations for compositional points from the audience's perspective. For example, it explains how an audience perceives the emotion of a piece of music. This allows the user to understand compositional points from multiple perspectives.

[0040] The annotation generation unit can visualize annotations for compositional points and provide visual explanations using diagrams and graphs. For example, the annotation generation unit visualizes annotations for compositional points and illustrates the phrasing of the main melody. For example, phrase breaks and emphasized parts are displayed in different colors. The annotation generation unit also visualizes annotations for compositional points and provides a diagram of harmonic progression. For example, arrows are used to indicate harmonic progression and explain the role of each chord. The annotation generation unit also visualizes annotations for compositional points and illustrates the relationship with the secondary melody in a graph. For example, lines are used to connect how the secondary melody complements the main melody. This allows the user to visually understand the compositional points.

[0041] The annotation generation unit can translate annotations for compositional points into different languages ​​to accommodate international users. For example, the annotation generation unit uses a generation AI to translate annotations for compositional points into English and provide them to English-speaking users. For example, it explains the characteristics of the main melody and points of harmonic progression in English. The annotation generation unit can also translate annotations for compositional points into French and provide them to French-speaking users. For example, it explains the relationship with the secondary melody in French. The annotation generation unit can also translate annotations for compositional points into Chinese and provide them to Chinese-speaking users. For example, it explains how to express harmonic progression in Chinese. This makes it possible to accommodate international users.

[0042] The recommendation unit can make recommendations not only based on a similar atmosphere to the generated song, but also taking into consideration technical difficulty and playing time. For example, the recommendation unit makes recommendations not only based on a similar atmosphere to the song generated by the generation AI, but also taking into consideration technical difficulty. For example, it can suggest easy songs for beginners. The recommendation unit can also make recommendations not only based on a similar atmosphere to the song generated by the generation AI, but also taking into consideration playing time. For example, it can suggest songs with short playing times. The recommendation unit can also make recommendations not only based on a similar atmosphere to the song generated by the generation AI, but also taking into consideration technical difficulty and playing time overall. For example, it can suggest songs with short playing times for beginners. This makes it possible to make recommendations taking into consideration technical difficulty and playing time.

[0043] The recommendation unit can also provide an explanation of the historical background and composer's intentions of the recommended song. For example, the recommendation unit provides an explanation of the historical background of the song recommended by the generation AI. For example, it explains the historical background of the time when Bach's music was composed and the characteristics of the music of that era. The recommendation unit also provides an explanation of the composer's intentions of the song recommended by the generation AI. For example, it explains Beethoven's intentions and emotions when he composed a particular song. The recommendation unit also provides a comprehensive explanation of the historical background and composer's intentions of the song recommended by the generation AI. For example, it explains the historical background of the time when Chopin's music was composed and Chopin's intentions. This allows the user to understand the historical background and composer's intentions of the recommended song.

[0044] The recommendation unit can also display other users' ratings and comments on the recommended songs for reference. The recommendation unit, for example, displays other users' ratings on the songs recommended by the generation AI. For example, it indicates how other users have rated the songs using the number of stars. The recommendation unit also displays other users' comments on the songs recommended by the generation AI. For example, it displays what other users think of the songs. The recommendation unit also comprehensively displays other users' ratings and comments on the songs recommended by the generation AI. For example, it displays the number of stars and comments together. This allows other users to refer to the ratings and comments.

[0045] The recommendation unit can propose different performance versions of the recommended song. For example, the recommendation unit proposes versions with different instrumental arrangements of the song recommended by the generation AI. For example, it proposes versions for piano solo, string quartet, orchestra, etc. The recommendation unit also proposes different arrangement versions of the song recommended by the generation AI. For example, it proposes arrangements in classical style, jazz style, pop style, etc. The recommendation unit also comprehensively proposes different performance versions of the song recommended by the generation AI. For example, it proposes versions with different instrumental arrangements and arrangements together. This makes it possible to propose different performance versions.

[0046] The completion unit can propose multiple different harmonic progressions and arrangements based on the user's melody. In the completion unit, for example, the generation AI proposes multiple different harmonic progressions based on the melody input by the user. For example, it proposes major and minor harmonic progressions for the same melody. In addition, in the completion unit, the generation AI proposes multiple different arrangements based on the melody input by the user. For example, it proposes classical, jazz, and pop arrangements for the same melody. In addition, in the completion unit, the generation AI proposes multiple different instrumental arrangements based on the melody input by the user. For example, it proposes piano solo, string quartet, and orchestral arrangements for the same melody. In this way, multiple different harmonic progressions and arrangements can be proposed based on the user's melody.

[0047] The completion unit can generate songs of different genres and styles based on the user's melody. For example, the completion unit generates songs of different genres, such as classical, jazz, and pop, based on the melody input by the user using a generation AI. For example, the completion unit generates classical and jazz songs for the same melody. The completion unit also generates songs of different era styles, such as baroque, romantic, and contemporary music, based on the melody input by the user. For example, the completion unit generates baroque and contemporary music for the same melody. The completion unit also generates songs of different cultural musical styles based on the melody input by the user using a generation AI. For example, the completion unit generates Western and Asian music for the same melody. This allows songs of different genres and styles to be generated based on the user's melody.

[0048] The completion unit can suggest different instrumental arrangements and arrangements based on the user's melody. For example, the completion unit allows the generation AI to suggest different instrumental arrangements, such as piano solo, string quartet, or orchestra, based on the melody input by the user. For example, it may suggest piano solo and orchestral arrangements for the same melody. The completion unit also allows the generation AI to suggest different arrangements, such as classical, jazz, and pop, based on the melody input by the user. For example, it may suggest classical and jazz arrangements for the same melody. The completion unit also allows the generation AI to suggest musical styles from different cultural spheres based on the melody input by the user. For example, it may suggest Western and Asian styles for the same melody. This allows different instrumental arrangements and arrangements to be suggested based on the user's melody.

[0049] The completion unit can fuse musical styles from different cultural spheres based on the user's melody. For example, the completion unit generates a song that fuses Western music and African rhythms based on a melody input by the user using a generation AI. For example, the generation AI adds a Western harmonic progression and African rhythm percussion to the same melody. The completion unit also generates a song that fuses classical music and Asian music based on a melody input by the user. For example, the generation AI adds a classical harmonic progression and Asian instruments to the same melody. The completion unit also generates a song that fuses jazz music and Latin music based on a melody input by the user. For example, the generation AI adds a jazz harmonic progression and Latin rhythm to the same melody. This makes it possible to generate a song that fuses musical styles from different cultural spheres based on the user's melody.

[0050] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0051] The condition specification unit enables the generation AI to compose hybrid compositions that combine musical styles from different cultural spheres based on user-specified parameters. For example, the generation AI generates a song that combines Western classical music and African rhythms based on user-specified conditions. For example, African rhythm percussion is added to a classical harmonic progression. The condition specification unit also enables the generation AI to generate a song that combines elements of traditional Asian music and jazz based on user-specified conditions. For example, jazz improvisation is added to a melody using traditional Asian instruments. The condition specification unit also enables the generation AI to generate a song that combines elements of Latin music and rock based on user-specified conditions. For example, a rock guitar riff is added to a Latin rhythm. This allows the user to generate new songs that combine musical styles from different cultural spheres.

[0052] The condition specification unit enables the generation AI to compose hybrid compositions that combine musical styles from different eras based on parameters specified by the user. For example, the generation AI generates a piece that combines elements of Baroque and contemporary music based on user-specified conditions. For example, adding contemporary electronic sounds to a Baroque harmonic progression. The condition specification unit also enables the generation AI to generate a piece that combines elements of Romantic and jazz based on user-specified conditions. For example, adding jazz improvisation to a Romantic melody line. The condition specification unit also enables the generation AI to generate a piece that combines elements of Classical and pop music based on user-specified conditions. For example, adding pop rhythms to a classical form. This allows the user to generate new pieces that combine musical styles from different eras.

[0053] The condition specification unit allows the generation AI to propose different instrumental arrangements and arrangements based on parameters specified by the user. For example, the generation AI proposes different instrumental arrangements such as piano solo, string quartet, and orchestra based on conditions specified by the user. For example, a Bach-style piece may be proposed to be performed by a string quartet. The condition specification unit also allows the generation AI to propose different arrangements based on conditions specified by the user. For example, the unit may propose arranging the same melody in a jazz, classical, or pop style. The condition specification unit also allows the generation AI to propose different instrument combinations based on conditions specified by the user. For example, the unit proposes combinations such as piano and flute, guitar and violin, etc. This allows the user to propose different instrumental arrangements and arrangements.

[0054] The music generation unit can incorporate not only the style of a specified composer, but also the styles of other composers who influenced that composer. For example, when the generation AI generates a Bach-style piece, it can also incorporate the styles of Vivaldi and Telemann, who influenced Bach. For example, it can add Vivaldi-style rhythms to a Bach-style harmonic progression. When the generation AI generates a Beethoven-style piece, it can also incorporate the styles of Mozart and Haydn, who influenced Beethoven. For example, it can add Mozart-style ornaments to a Beethoven-style melody line. When the generation AI generates a Chopin-style piece, it can also incorporate the styles of Bach and Mozart, who influenced Chopin. For example, it can add Bach-style counterpoint to a Chopin-style piano piece. This makes it possible to generate music that incorporates not only the style of a composer specified by the user, but also the styles of other composers who influenced that composer.

[0055] The music generation unit can generate new music by combining styles from different periods of a specified composer. For example, when the generation AI generates a Bach-style piece, it combines the styles of Bach's early, middle, and late works. For example, it combines an early fugue form with a late cantata form. When the generation AI generates a Beethoven-style piece, it combines the styles of Beethoven's early, middle, and late works. For example, it combines an early sonata form with a late symphony form. When the generation AI generates a Mozart-style piece, it combines the styles of Mozart's early, middle, and late works. For example, it combines an early opera form with a late piano concerto form. This allows the generation of new music by combining styles from different periods of a composer specified by the user.

[0056] The music generation unit can blend musical styles from different cultural spheres in addition to the style of a specified composer. For example, when the generation AI generates a Bach-style piece, it blends African rhythms into the Bach style. For example, it adds African rhythm percussion to a Bach-style harmonic progression. When the generation AI generates a Beethoven-style piece, it blends Asian musical elements into the Beethoven style. For example, it adds traditional Chinese instruments to a Beethoven-style melody line. When the generation AI generates a Chopin-style piece, it blends Latin music elements into the Chopin style. For example, it adds samba rhythms to a Chopin-style piano piece. This makes it possible to generate a piece that blends musical styles from different cultural spheres in addition to the style of a composer specified by the user.

[0057] The processing flow of the first embodiment will be briefly explained below.

[0058] Step 1: The condition specification unit receives parameters specified by the user. For example, the user can specify parameters such as major or minor key, time signature, tempo, composition format, song length, and style. Step 2: The music generation unit generates a music piece based on the parameters received by the condition specification unit. For example, when generating a Bach-style music piece, the generation AI uses a model that has learned Bach's compositional style to generate Bach-style harmonic progressions and melody lines. Step 3: The annotation generator generates annotations for the compositional points of the song generated by the song generator. For example, the AI ​​generates annotations that explain the characteristics of the main melody, points in harmonic progression, and the relationship with the secondary melody. Step 4: The recommendation unit recommends actual music pieces that have a similar atmosphere to the music piece generated by the music generation unit. For example, if the generated music piece is Bach-like, the recommendation unit recommends actual music pieces by Bach. Step 5: The completion unit completes the song based on the user's original melody. For example, based on a melody created by the user, the generation AI adds harmonic progressions, counter melodies, and bass parts to complete the song.

[0059] (Example 2) The composition support system according to the embodiment of the present invention automatically generates compositions based on parameters specified by the user, generates annotations for compositional points, recommends actual songs, and complements compositions based on the user's own melodies. This allows even beginners to easily enjoy composing.

[0060] A composition assistance system according to an embodiment includes a condition specification unit, a music generation unit, an annotation generation unit, a recommendation unit, and a completion unit. The condition specification unit receives parameters specified by a user. For example, the user can specify parameters such as major or minor key, time signature, tempo, composition format, music length, and a specific style. The music generation unit generates a music piece based on the parameters received by the condition specification unit. For example, when generating a Bach-style music piece, the generation AI uses a model that has learned Bach's compositional style to generate a Bach-style harmonic progression and melody line. The annotation generation unit generates annotations for the compositional points of the music piece generated by the music generation unit. For example, the generation AI generates annotations that explain the characteristics of the main melody, key points in the harmonic progression, and the relationship with the secondary melody. The recommendation unit recommends actual music pieces that have a similar atmosphere to the music piece generated by the music generation unit. For example, if the generated music piece is Bach-style, the recommendation unit recommends an actual Bach piece. The completion unit completes a song based on a user's original melody. For example, based on a melody created by the user, the generation AI adds a harmonic progression, a counter melody, and a bass part to complete the song. This allows the composition support system according to the embodiment to make it easy for even beginners to enjoy composing. For example, if a user wants to create a Bach-style 4 / 4 song, the user simply specifies the conditions, and the generation AI automatically generates a Bach-style song and provides annotations on compositional points and actual songs for reference. Users can also complete a song based on a melody they created themselves.

[0061] The condition specification unit can automatically present reference materials for the generation AI to draw inspiration from works by past composers based on parameters specified by the user. For example, the condition specification unit can cause the generation AI to automatically search for works by Bach or Beethoven based on user-specified conditions and present them as reference materials. For example, if a user wants to create a Bach-style piece, the condition specification unit can list Bach's representative works. The condition specification unit can also cause the generation AI to analyze the works of a specific composer based on user-specified parameters and provide the analysis results as reference materials. For example, the condition specification unit can analyze Beethoven's sonata-form works and show their structure. The condition specification unit can also cause the generation AI to provide reference materials based on music theory based on user-specified conditions. For example, if a user wants to create a 4 / 4 Allegro piece, the condition specification unit can present information on 4 / 4 rhythm patterns and Allegro tempos. This allows the user to draw inspiration from the works of past composers.

[0062] The condition specification unit enables the generation AI to create hybrid compositions that combine multiple different compositional styles based on user-specified parameters. For example, the condition specification unit enables the generation AI to generate a piece that combines Bach-style and Beethoven-style elements based on user-specified conditions. For example, a Beethoven-style melody line is added to a Bach-style harmonic progression. The condition specification unit also enables the generation AI to generate a hybrid piece that combines classical and jazz elements based on user-specified conditions. For example, jazz improvisation is incorporated into a classical form. The condition specification unit also enables the generation AI to generate a piece that combines compositional styles from different eras based on user-specified conditions. For example, a Romantic melody is added to a Baroque-style harmonic progression. This allows the user to create new pieces that combine multiple compositional styles.

[0063] The condition specification unit can use the emotion estimation function to suggest optimal composition conditions based on the user's current emotional state. For example, the condition specification unit analyzes the user's emotional state in real time, and if the user is in a positive emotional state, suggests a song in a bright major key. For example, if the user is feeling happy, suggests a song in a major key with an allegro tempo. The condition specification unit also analyzes the user's emotional state, and if the user is in a negative emotional state, suggests a song in a calm minor key. For example, if the user is feeling sad, suggests a song in a minor key with an adagio tempo. The condition specification unit also analyzes the user's emotional state, and suggests a composition style that matches that emotion. For example, if the user wants to relax, suggests a song in rondo form. This makes it possible to suggest optimal composition conditions based on the user's emotional state.

[0064] The condition specification unit provides a visual selection interface for the user when specifying composition conditions, enabling intuitive parameter setting. The condition specification unit, for example, provides a visual selection interface for the user, allowing parameters such as major or minor key, time signature, and tempo to be set by drag and drop. For example, sliders or drop-down menus are used. The condition specification unit also provides a visual selection interface for the user, allowing the user to select the composition form and length of the piece using icons or graphics. For example, by clicking an icon for sonata form or rondo form. The condition specification unit also provides a visual selection interface for the user, allowing the user to select a certain style using an image or sample sound source. For example, by clicking an image of Bach or Beethoven. This allows the user to visually set composition conditions.

[0065] The condition specification unit allows the generation AI to propose different instrumental arrangements and arrangements based on parameters specified by the user. For example, the condition specification unit allows the generation AI to propose different instrumental arrangements, such as piano solo, string quartet, or orchestra, based on conditions specified by the user. For example, the condition specification unit suggests playing a Bach-style piece with a string quartet. The condition specification unit also allows the generation AI to propose different arrangements based on conditions specified by the user. For example, the condition specification unit suggests arranging the same melody in a jazz, classical, or pop style. The condition specification unit also allows the generation AI to propose different instrument combinations based on conditions specified by the user. For example, the condition specification unit suggests combinations such as piano and flute, guitar and violin, etc. This allows the user to propose different instrumental arrangements and arrangements.

[0066] The condition specification unit can use the emotion estimation function to analyze the emotional tendencies of songs created by the user in the past and propose new composition conditions based on the results. For example, the condition specification unit analyzes the emotional tendencies of songs created by the user in the past, and if there are many positive emotions, proposes songs in bright major keys. For example, if the past songs have a joyful emotion, proposes songs in major keys with an allegro tempo. The condition specification unit also analyzes the emotional tendencies of songs created by the user in the past and if there are many negative emotions, proposes songs in calm minor keys. For example, if the past songs have a sad emotion, proposes songs in minor keys with an adagio tempo. The condition specification unit also analyzes the emotional tendencies of songs created by the user in the past and proposes a composition style that matches the emotion. For example, if the past songs are relaxing, proposes a rondo-form song. This makes it possible to propose new composition conditions based on the user's past composition tendencies.

[0067] The music generation unit can incorporate not only the style of a specified composer, but also the styles of other composers who influenced that composer. For example, when the generation AI generates a Bach-style piece, the music generation unit also incorporates the styles of Vivaldi and Telemann, who influenced Bach. For example, Vivaldi-style rhythms are added to Bach-style harmonic progressions. When the generation AI generates a Beethoven-style piece, the music generation unit also incorporates the styles of Mozart and Haydn, who influenced Beethoven. For example, Mozart-style ornaments are added to Beethoven-style melody lines. When the generation AI generates a Chopin-style piece, the music generation unit also incorporates the styles of Bach and Mozart, who influenced Chopin. For example, Bach-style counterpoint is added to a Chopin-style piano piece. This makes it possible to generate music that incorporates not only the style of a composer specified by the user, but also the styles of other composers who influenced that composer.

[0068] The music generation unit can generate new music by combining styles from different periods of a specified composer. For example, when the generation AI generates a Bach-style piece, the music generation unit combines the styles of Bach's early, middle, and late works. For example, it combines an early fugue form with a late cantata form. When the generation AI generates a Beethoven-style piece, the music generation unit combines the styles of Beethoven's early, middle, and late works. For example, it combines an early sonata form with a late symphony form. When the generation AI generates a Mozart-style piece, the music generation unit combines the styles of Mozart's early, middle, and late works. For example, it combines an early opera form with a late piano concerto form. This allows the generation of new music by combining styles from different periods of a composer specified by the user.

[0069] The music generation unit can use the emotion estimation function to fine-tune the composer's style based on the user's emotion and generate a more emotionally appealing piece. For example, the music generation unit analyzes the user's emotional state, and if the user is in a positive emotional state, adds a bright harmonic progression to a Bach-style piece. For example, for a user who is feeling happy, adds major harmonies to a Bach-style piece. The music generation unit also analyzes the user's emotional state, and if the user is in a negative emotional state, adds a calm melody line to a Beethoven-style piece. For example, for a user who is feeling sad, adds a minor melody to a Beethoven-style piece. The music generation unit also analyzes the user's emotional state and fine-tunes the composer's style to match the emotion. For example, for a user who wants to relax, adds a gentle rhythm to a Chopin-style piece. In this way, the composer's style can be fine-tuned based on the user's emotion, generating a more emotionally appealing piece.

[0070] The music generation unit can blend musical styles from different cultural spheres in addition to the style of a specified composer. For example, when the generation AI generates a Bach-style piece, the music generation unit blends African rhythms into the Bach style. For example, African rhythm percussion is added to a Bach-style harmonic progression. When the generation AI generates a Beethoven-style piece, the music generation unit blends Asian musical elements into the Beethoven style. For example, traditional Chinese instruments are added to a Beethoven-style melody line. When the generation AI generates a Chopin-style piece, the music generation unit blends Latin music elements into the Chopin style. For example, samba rhythms are added to a Chopin-style piano piece. This makes it possible to generate a piece that blends musical styles from different cultural spheres in addition to the style of a composer specified by the user.

[0071] The music generation unit can arrange the style of a specified composer in a modern style and incorporate modern elements. For example, when the generation AI generates a Bach-style piece, the music generation unit adds elements of modern electronic music to the Bach style. For example, it adds synthesizer tones to a Bach-style harmonic progression. Furthermore, when the generation AI generates a Beethoven-style piece, the music generation unit adds elements of modern pop music to the Beethoven style. For example, it adds pop rhythms to a Beethoven-style melody line. Furthermore, when the generation AI generates a Chopin-style piece, the music generation unit adds elements of modern jazz to the Chopin style. For example, it adds jazz improvisation to a Chopin-style piano piece. In this way, the style of a composer specified by the user can be arranged in a modern style and a piece incorporating modern elements can be generated.

[0072] The music generation unit can use the emotion estimation function to analyze the style of music that the user liked in the past and propose a new composition style based on that. For example, the music generation unit analyzes the style of music that the user liked in the past and proposes a new composition style based on that style. For example, if the user likes Bach's music, it proposes a new Bach-like style. The music generation unit also analyzes the emotional tendencies of the music that the user liked in the past and proposes a new composition style based on that emotion. For example, if the user wants to relax, it proposes a calm Chopin-like style. The music generation unit also analyzes the rhythms and melody lines of the music that the user liked in the past and proposes a new composition style based on that. For example, if the user likes jazz rhythms, it proposes a new jazz-like style. In this way, it is possible to propose a new composition style based on the style of music that the user liked in the past.

[0073] The annotation generation unit can also explain specific performance techniques and methods of expression in addition to annotations of compositional points. For example, the generation AI may explain performance techniques for the main melody in addition to annotations of compositional points. For example, it may detail the phrasing and articulation methods of the main melody. Furthermore, the annotation generation unit may explain methods for expressing harmonic progression in addition to annotations of compositional points. For example, it may detail the dynamics and tempo changes of the harmonic progression. Furthermore, the annotation generation unit may explain the relationship with the secondary melody in addition to annotations of compositional points. For example, it may detail how the secondary melody complements the main melody. This allows the user to understand specific performance techniques and methods of expression.

[0074] The annotation generation unit can provide annotations for compositional points from multiple perspectives. For example, the generation AI provides annotations for compositional points from the composer's perspective. For example, it explains the composer's intention in choosing a particular harmonic progression. The annotation generation unit also provides annotations for compositional points from the performer's perspective. For example, it explains how a performer expresses the main melody. The annotation generation unit also provides annotations for compositional points from the audience's perspective. For example, it explains how an audience perceives the emotion of a piece of music. This allows the user to understand compositional points from multiple perspectives.

[0075] The annotation generation unit uses the emotion estimation function to adjust the content of the annotation based on the user's emotion, thereby providing a more empathetic explanation. The annotation generation unit, for example, analyzes the user's emotional state, and if the user is in a positive emotional state, adds cheerful expressions to the annotation. For example, for a user who is feeling happy, it adds positive words to the annotation. The annotation generation unit also analyzes the user's emotional state, and if the user is in a negative emotional state, it adds calming expressions to the annotation. For example, for a user who is feeling sad, it adds calming words to the annotation. The annotation generation unit also analyzes the user's emotional state, and provides annotations that match the emotion. For example, for a user who wants to relax, it adds expressions that have a relaxing effect to the annotation. In this way, the content of the annotation can be adjusted based on the user's emotion, providing a more empathetic explanation.

[0076] The annotation generation unit can visualize annotations for compositional points and provide visual explanations using diagrams and graphs. For example, the annotation generation unit visualizes annotations for compositional points and illustrates the phrasing of the main melody. For example, phrase breaks and emphasized parts are displayed in different colors. The annotation generation unit also visualizes annotations for compositional points and provides a diagram of harmonic progression. For example, arrows are used to indicate harmonic progression and explain the role of each chord. The annotation generation unit also visualizes annotations for compositional points and illustrates the relationship with the secondary melody in a graph. For example, lines are used to connect how the secondary melody complements the main melody. This allows the user to visually understand the compositional points.

[0077] The annotation generation unit can translate annotations for compositional points into different languages ​​to accommodate international users. For example, the annotation generation unit uses a generation AI to translate annotations for compositional points into English and provide them to English-speaking users. For example, it explains the characteristics of the main melody and points of harmonic progression in English. The annotation generation unit can also translate annotations for compositional points into French and provide them to French-speaking users. For example, it explains the relationship with the secondary melody in French. The annotation generation unit can also translate annotations for compositional points into Chinese and provide them to Chinese-speaking users. For example, it explains how to express harmonic progression in Chinese. This makes it possible to accommodate international users.

[0078] The annotation generation unit can use the emotion estimation function to analyze the explanation style that is easiest for the user to understand and generate annotations based on that. For example, the annotation generation unit analyzes the user's emotional state, and if the user is in a positive emotional state, adds cheerful expressions to the annotation. For example, for a user who is feeling happy, it adds positive words to the annotation. Furthermore, the annotation generation unit analyzes the user's emotional state, and if the user is in a negative emotional state, it adds calming expressions to the annotation. For example, for a user who is feeling sad, it adds gentle words to the annotation. Furthermore, the annotation generation unit analyzes the user's emotional state and provides annotations that match that emotion. For example, for a user who wants to relax, it adds expressions that have a relaxing effect to the annotation. This makes it possible to provide annotations in an explanation style that is easiest for the user to understand.

[0079] The recommendation unit can make recommendations not only based on a similar atmosphere to the generated song, but also taking into consideration technical difficulty and playing time. For example, the recommendation unit makes recommendations not only based on a similar atmosphere to the song generated by the generation AI, but also taking into consideration technical difficulty. For example, it can suggest easy songs for beginners. The recommendation unit can also make recommendations not only based on a similar atmosphere to the song generated by the generation AI, but also taking into consideration playing time. For example, it can suggest songs with short playing times. The recommendation unit can also make recommendations not only based on a similar atmosphere to the song generated by the generation AI, but also taking into consideration technical difficulty and playing time overall. For example, it can suggest songs with short playing times for beginners. This makes it possible to make recommendations taking into consideration technical difficulty and playing time.

[0080] The recommendation unit can also provide an explanation of the historical background and composer's intentions of the recommended song. For example, the recommendation unit provides an explanation of the historical background of the song recommended by the generation AI. For example, it explains the historical background of the time when Bach's music was composed and the characteristics of the music of that era. The recommendation unit also provides an explanation of the composer's intentions of the song recommended by the generation AI. For example, it explains Beethoven's intentions and emotions when he composed a particular song. The recommendation unit also provides a comprehensive explanation of the historical background and composer's intentions of the song recommended by the generation AI. For example, it explains the historical background of the time when Chopin's music was composed and Chopin's intentions. This allows the user to understand the historical background and composer's intentions of the recommended song.

[0081] The recommendation unit can use the emotion estimation function to recommend songs that the user can most emotionally empathize with based on the user's emotions. For example, the recommendation unit analyzes the user's emotional state and recommends cheerful songs if the user is in a positive emotional state. For example, it proposes cheerful songs in major keys to a user who is feeling happy. The recommendation unit also analyzes the user's emotional state and recommends calm songs if the user is in a negative emotional state. For example, it proposes calm songs in minor keys to a user who is feeling sad. The recommendation unit also analyzes the user's emotional state and recommends songs that match that emotion. For example, it proposes songs with a relaxing effect to a user who wants to relax. This makes it possible to recommend songs that the user can most emotionally empathize with based on the user's emotions.

[0082] The recommendation unit can also display other users' ratings and comments on the recommended songs for reference. The recommendation unit, for example, displays other users' ratings on the songs recommended by the generation AI. For example, it indicates how other users have rated the songs using the number of stars. The recommendation unit also displays other users' comments on the songs recommended by the generation AI. For example, it displays what other users think of the songs. The recommendation unit also comprehensively displays other users' ratings and comments on the songs recommended by the generation AI. For example, it displays the number of stars and comments together. This allows other users to refer to the ratings and comments.

[0083] The recommendation unit can propose different performance versions of the recommended song. For example, the recommendation unit proposes versions with different instrumental arrangements of the song recommended by the generation AI. For example, it proposes versions for piano solo, string quartet, orchestra, etc. The recommendation unit also proposes different arrangement versions of the song recommended by the generation AI. For example, it proposes arrangements in classical style, jazz style, pop style, etc. The recommendation unit also comprehensively proposes different performance versions of the song recommended by the generation AI. For example, it proposes versions with different instrumental arrangements and arrangements together. This makes it possible to propose different performance versions.

[0084] The recommendation unit uses the emotion estimation function to analyze the trends of songs that the user has liked in the past and can make new recommendations based on the results. For example, the recommendation unit analyzes the trends of songs that the user has liked in the past and recommends new songs based on those trends. For example, if the user likes Bach's music, it will suggest new Bach-style songs. The recommendation unit also analyzes the emotional trends of songs that the user has liked in the past and recommends new songs based on those emotions. For example, if the user wants to relax, it will suggest songs that have a relaxing effect. The recommendation unit also analyzes the rhythms and melody lines of songs that the user has liked in the past and recommends new songs based on those. For example, if the user likes jazz rhythms, it will suggest new jazz-style songs. This allows new recommendations to be made based on the trends of songs that the user has liked in the past.

[0085] The completion unit can propose multiple different harmonic progressions and arrangements based on the user's melody. In the completion unit, for example, the generation AI proposes multiple different harmonic progressions based on the melody input by the user. For example, it proposes major and minor harmonic progressions for the same melody. In addition, in the completion unit, the generation AI proposes multiple different arrangements based on the melody input by the user. For example, it proposes classical, jazz, and pop arrangements for the same melody. In addition, in the completion unit, the generation AI proposes multiple different instrumental arrangements based on the melody input by the user. For example, it proposes piano solo, string quartet, and orchestral arrangements for the same melody. In this way, multiple different harmonic progressions and arrangements can be proposed based on the user's melody.

[0086] The completion unit can generate songs of different genres and styles based on the user's melody. For example, the completion unit generates songs of different genres, such as classical, jazz, and pop, based on the melody input by the user using a generation AI. For example, the completion unit generates classical and jazz songs for the same melody. The completion unit also generates songs of different era styles, such as baroque, romantic, and contemporary music, based on the melody input by the user. For example, the completion unit generates baroque and contemporary music for the same melody. The completion unit also generates songs of different cultural musical styles based on the melody input by the user using a generation AI. For example, the completion unit generates Western and Asian music for the same melody. This allows songs of different genres and styles to be generated based on the user's melody.

[0087] The completion unit can use the emotion estimation function to suggest harmonic progressions and arrangements that are most emotionally relatable to the user based on their emotions. For example, the completion unit analyzes the user's emotional state, and if the user is in a positive emotional state, suggests a bright harmonic progression. For example, it suggests a major harmonic progression to a user who is feeling happy. The completion unit also analyzes the user's emotional state, and if the user is in a negative emotional state, suggests a calm harmonic progression. For example, it suggests a minor harmonic progression to a user who is feeling sad. The completion unit also analyzes the user's emotional state, and suggests an arrangement that matches that emotion. For example, it suggests a calm arrangement to a user who wants to relax. In this way, it is possible to suggest harmonic progressions and arrangements that are most emotionally relatable to the user based on their emotions.

[0088] The completion unit can suggest different instrumental arrangements and arrangements based on the user's melody. For example, the completion unit allows the generation AI to suggest different instrumental arrangements, such as piano solo, string quartet, or orchestra, based on the melody input by the user. For example, it may suggest piano solo and orchestral arrangements for the same melody. The completion unit also allows the generation AI to suggest different arrangements, such as classical, jazz, and pop, based on the melody input by the user. For example, it may suggest classical and jazz arrangements for the same melody. The completion unit also allows the generation AI to suggest musical styles from different cultural spheres based on the melody input by the user. For example, it may suggest Western and Asian styles for the same melody. This allows different instrumental arrangements and arrangements to be suggested based on the user's melody.

[0089] The completion unit can fuse musical styles from different cultural spheres based on the user's melody. For example, the completion unit generates a song that fuses Western music and African rhythms based on a melody input by the user using a generation AI. For example, the generation AI adds a Western harmonic progression and African rhythm percussion to the same melody. The completion unit also generates a song that fuses classical music and Asian music based on a melody input by the user. For example, the generation AI adds a classical harmonic progression and Asian instruments to the same melody. The completion unit also generates a song that fuses jazz music and Latin music based on a melody input by the user. For example, the generation AI adds a jazz harmonic progression and Latin rhythm to the same melody. This makes it possible to generate a song that fuses musical styles from different cultural spheres based on the user's melody.

[0090] The completion unit can use the emotion estimation function to analyze the emotional tendencies of songs created in the past by the user and complete a new melody based on that. The completion unit, for example, analyzes the emotional tendencies of songs created in the past by the user and completes a new melody based on that tendency. For example, if a past song has a positive emotion, it completes a bright melody. The completion unit also analyzes the emotional tendencies of songs created in the past by the user and suggests a new harmonic progression based on that tendency. For example, if a past song has a negative emotion, it suggests a calm harmonic progression. The completion unit also analyzes the emotional tendencies of songs created in the past by the user and suggests a new arrangement based on that tendency. For example, if a past song is relaxing, it suggests a calm arrangement. In this way, a new melody can be completed based on the emotional tendencies of songs created in the past by the user.

[0091] The completion unit can use the emotion estimation function to analyze the emotional tendencies of songs created in the past by the user and complete a new melody based on that. The completion unit, for example, analyzes the emotional tendencies of songs created in the past by the user and completes a new melody based on that tendency. For example, if a past song has a positive emotion, it completes a bright melody. The completion unit also analyzes the emotional tendencies of songs created in the past by the user and suggests a new harmonic progression based on that tendency. For example, if a past song has a negative emotion, it suggests a calm harmonic progression. The completion unit also analyzes the emotional tendencies of songs created in the past by the user and suggests a new arrangement based on that tendency. For example, if a past song is relaxing, it suggests a calm arrangement. In this way, a new melody can be completed based on the emotional tendencies of songs created in the past by the user.

[0092] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0093] The condition specification unit enables the generation AI to compose hybrid compositions that combine musical styles from different cultural spheres based on user-specified parameters. For example, the generation AI generates a song that combines Western classical music and African rhythms based on user-specified conditions. For example, African rhythm percussion is added to a classical harmonic progression. The condition specification unit also enables the generation AI to generate a song that combines elements of traditional Asian music and jazz based on user-specified conditions. For example, jazz improvisation is added to a melody using traditional Asian instruments. The condition specification unit also enables the generation AI to generate a song that combines elements of Latin music and rock based on user-specified conditions. For example, a rock guitar riff is added to a Latin rhythm. This allows the user to generate new songs that combine musical styles from different cultural spheres.

[0094] The condition specification unit can provide reference material based on music theory for the generation AI to elicit specific emotions based on parameters specified by the user. For example, based on conditions specified by the user, the generation AI presents a major harmonic progression and rhythmic pattern to elicit emotions of joy. Also, based on conditions specified by the user, the condition specification unit presents a minor harmonic progression and a slow tempo rhythmic pattern to elicit emotions of sadness. Furthermore, based on conditions specified by the user, the condition specification unit provides reference material based on music theory that has a relaxing effect. For example, a slow tempo and a gentle harmonic progression are presented. This allows the user to obtain reference material based on music theory for eliciting specific emotions.

[0095] The condition specification unit enables the generation AI to compose hybrid compositions that combine musical styles from different eras based on parameters specified by the user. For example, the generation AI generates a piece that combines elements of Baroque and contemporary music based on user-specified conditions. For example, adding contemporary electronic sounds to a Baroque harmonic progression. The condition specification unit also enables the generation AI to generate a piece that combines elements of Romantic and jazz based on user-specified conditions. For example, adding jazz improvisation to a Romantic melody line. The condition specification unit also enables the generation AI to generate a piece that combines elements of Classical and pop music based on user-specified conditions. For example, adding pop rhythms to a classical form. This allows the user to generate new pieces that combine musical styles from different eras.

[0096] The condition specification unit can use the emotion estimation function to suggest optimal composition conditions based on the user's emotional state. For example, the unit analyzes the user's emotional state in real time, and if the user is in a positive emotional state, suggests a song in a bright major key. For example, if the user is feeling happy, suggests a song in a major key with an allegro tempo. The condition specification unit also analyzes the user's emotional state, and if the user is in a negative emotional state, suggests a song in a calm minor key. For example, if the user is feeling sad, suggests a song in a minor key with an adagio tempo. The condition specification unit also analyzes the user's emotional state, and suggests a composition style that matches that emotion. For example, if the user wants to relax, suggests a song in rondo form. This makes it possible to suggest optimal composition conditions based on the user's emotional state.

[0097] The condition specification unit allows the generation AI to propose different instrumental arrangements and arrangements based on parameters specified by the user. For example, the generation AI proposes different instrumental arrangements such as piano solo, string quartet, and orchestra based on conditions specified by the user. For example, a Bach-style piece may be proposed to be performed by a string quartet. The condition specification unit also allows the generation AI to propose different arrangements based on conditions specified by the user. For example, the unit may propose arranging the same melody in a jazz, classical, or pop style. The condition specification unit also allows the generation AI to propose different instrument combinations based on conditions specified by the user. For example, the unit proposes combinations such as piano and flute, guitar and violin, etc. This allows the user to propose different instrumental arrangements and arrangements.

[0098] The condition specification unit can use the emotion estimation function to analyze the emotional tendencies of songs created by the user in the past and suggest new composition conditions based on the results. For example, if the emotional tendencies of songs created by the user in the past are analyzed and there are many positive emotions, songs in bright major keys are suggested. For example, if the past songs have a joyful emotion, songs in major keys with an allegro tempo are suggested. The condition specification unit can also analyze the emotional tendencies of songs created by the user in the past and there are many negative emotions, songs in calm minor keys are suggested. For example, if the past songs have a sad emotion, songs in minor keys with an adagio tempo are suggested. The condition specification unit can also analyze the emotional tendencies of songs created by the user in the past and suggest a composition style that matches the emotion. For example, if the past songs are relaxing, songs in rondo form are suggested. This makes it possible to suggest new composition conditions based on the user's past composition tendencies.

[0099] The music generation unit can incorporate not only the style of a specified composer, but also the styles of other composers who influenced that composer. For example, when the generation AI generates a Bach-style piece, it can also incorporate the styles of Vivaldi and Telemann, who influenced Bach. For example, it can add Vivaldi-style rhythms to a Bach-style harmonic progression. When the generation AI generates a Beethoven-style piece, it can also incorporate the styles of Mozart and Haydn, who influenced Beethoven. For example, it can add Mozart-style ornaments to a Beethoven-style melody line. When the generation AI generates a Chopin-style piece, it can also incorporate the styles of Bach and Mozart, who influenced Chopin. For example, it can add Bach-style counterpoint to a Chopin-style piano piece. This makes it possible to generate music that incorporates not only the style of a composer specified by the user, but also the styles of other composers who influenced that composer.

[0100] The music generation unit can generate new music by combining styles from different periods of a specified composer. For example, when the generation AI generates a Bach-style piece, it combines the styles of Bach's early, middle, and late works. For example, it combines an early fugue form with a late cantata form. When the generation AI generates a Beethoven-style piece, it combines the styles of Beethoven's early, middle, and late works. For example, it combines an early sonata form with a late symphony form. When the generation AI generates a Mozart-style piece, it combines the styles of Mozart's early, middle, and late works. For example, it combines an early opera form with a late piano concerto form. This allows the generation of new music by combining styles from different periods of a composer specified by the user.

[0101] The music generation unit can use the emotion estimation function to fine-tune the composer's style based on the user's emotion and generate a more emotionally appealing piece. For example, the music generation unit analyzes the user's emotional state, and if the user is in a positive emotional state, adds a bright harmonic progression to a Bach-style piece. For example, if the user is feeling happy, adds major harmonies to a Bach-style piece. The music generation unit also analyzes the user's emotional state, and if the user is in a negative emotional state, adds a calm melody line to a Beethoven-style piece. For example, if the user is feeling sad, adds a minor melody to a Beethoven-style piece. The music generation unit also analyzes the user's emotional state and fine-tunes the composer's style to match the emotion. For example, if the user wants to relax, adds a gentle rhythm to a Chopin-style piece. In this way, the composer's style can be fine-tuned based on the user's emotion, generating a more emotionally appealing piece.

[0102] The music generation unit can blend musical styles from different cultural spheres in addition to the style of a specified composer. For example, when the generation AI generates a Bach-style piece, it blends African rhythms into the Bach style. For example, it adds African rhythm percussion to a Bach-style harmonic progression. When the generation AI generates a Beethoven-style piece, it blends Asian musical elements into the Beethoven style. For example, it adds traditional Chinese instruments to a Beethoven-style melody line. When the generation AI generates a Chopin-style piece, it blends Latin music elements into the Chopin style. For example, it adds samba rhythms to a Chopin-style piano piece. This makes it possible to generate a piece that blends musical styles from different cultural spheres in addition to the style of a composer specified by the user.

[0103] The processing flow of the second embodiment will be briefly explained below.

[0104] Step 1: The condition specification unit receives parameters specified by the user. For example, the user can specify parameters such as major or minor key, time signature, tempo, composition format, song length, and style. Step 2: The music generation unit generates a music piece based on the parameters received by the condition specification unit. For example, when generating a Bach-style music piece, the generation AI uses a model that has learned Bach's compositional style to generate Bach-style harmonic progressions and melody lines. Step 3: The annotation generator generates annotations for the compositional points of the song generated by the song generator. For example, the AI ​​generates annotations that explain the characteristics of the main melody, points in harmonic progression, and the relationship with the secondary melody. Step 4: The recommendation unit recommends actual music pieces that have a similar atmosphere to the music piece generated by the music generation unit. For example, if the generated music piece is Bach-like, the recommendation unit recommends actual music pieces by Bach. Step 5: The completion unit completes the song based on the user's original melody. For example, based on a melody created by the user, the generation AI adds harmonic progressions, counter melodies, and bass parts to complete the song.

[0105] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0106] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> Examples of generative AIs include the data generation model 58, such as a neural network model (e.g., a neural network model), and a neural network model (e.g., a neural network model). The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating speech, text data indicating text, and image data indicating an image is also input to the data generation model 58. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0107] Furthermore, the processing by the data processing system 10 described above is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0108] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0109] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0110] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0111] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0112] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0113] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0114] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0115] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0116] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0117] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0118] In the smart glasses 214, the specific processing is performed by the processor 46. A specific processing program 60 is stored in the storage 50. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0119] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0120] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0121] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AI other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0122] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, etc., and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0123] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0124] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0125] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0126] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0127] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0128] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0129] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0130] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0131] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0132] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0133] In the headset type terminal 314, the specific processing is performed by the processor 46. A specific processing program 60 is stored in the storage 50. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the headset type terminal 314 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0134] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0135] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0136] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AI other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0137] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, etc., and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0138] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0139] 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0140] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0141] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0142] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0143] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0144] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0145] The control object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0146] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0147] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0148] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0149] In the robot 414, the specific processing is performed by the processor 46. A specific processing program 60 is stored in the storage 50. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. The robot 414 also has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0150] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0151] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0152] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AI other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0153] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0154] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0155] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion encompasses both emotions and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0156] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0157] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0158] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. Emotions can also be created for robots, cars, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on speech emotion recognition and brain physiological signal analysis systems for emotions, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the area called "reaction," where sensation is dominant. The right half of the emotion map lists emotions belonging to the area called "situation," where situational awareness is dominant.

[0159] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[0160] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[0161] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers including computer 22.

[0162] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0163] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0164] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0165] The hardware resource for executing a specific process can be any of the following types of processors: A processor, for example, is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. A processor also includes a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[0166] The hardware resource that executes the specific process may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0167] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0168] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[0169] In the above example, the first to fourth embodiments have been described separately, but some or all of these embodiments may be combined. The smart device 14, smart glasses 214, headset terminal 314, and robot 414 are merely examples, and they may be combined, or other devices may be used. In the above example, the first and second embodiments have been described separately, but they may be combined.

[0170] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0171] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference. [Explanation of symbols]

[0172] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. a condition specification section that receives parameters specified by a user; a music generating unit that generates a music piece based on the parameters received by the condition specifying unit; an annotation generating unit that generates annotations of composition points of the music piece generated by the music generating unit; a recommendation unit that recommends actual music that has a similar atmosphere to the music generated by the music generation unit; A completion unit that completes a song based on a user's original melody. A system characterized by:

2. The condition specification unit Based on the parameters specified by the user, the generative AI automatically suggests reference materials for inspiration from past composers' works.

2. The system of claim 1.

3. The condition specification unit Based on the parameters specified by the user, the generation AI creates a hybrid composition that combines multiple different composition styles.

2. The system of claim 1.

4. The condition specification unit Suggesting optimal composition conditions based on the user's current emotional state 2. The system of claim 1.

5. The condition specification unit When specifying composition conditions, an interface is provided that allows users to visually select parameters, enabling intuitive setting of the parameters.

2. The system of claim 1.

6. The condition specification unit The generating AI proposes different instrument arrangements and arrangements based on the parameters specified by the user.

2. The system of claim 1.

7. The condition specification unit Analyzes the emotional trends of songs created by users in the past and suggests new composition conditions based on those trends.

2. The system of claim 1.

Images