Music control system and method based on 7CM theory

Abstract

Traditional music theory contains many exceptions, making it difficult to directly apply its principles. Furthermore, interactively changing tonality and chord progressions in response to user behavior, such as game situations, presents challenges. Additionally, a lack of information management technology at the tonal and functional levels makes it difficult to reuse elements like tonality and progression during music production. [Solution] Based on the 7CM theory (a specific music theory system), this system decomposes music into a framework of intervals and luminosity values, and manages functions and gravitational forces within tonalities using an intuitive, unique UI, thereby providing a music control system that enables easy music generation and editing even without specialized knowledge of music theory. It also enables interactive music production that responds to user behavior, supporting the creation of immersive music for experiential content such as games. Furthermore, it enables efficient reuse through information management for each key and functional tone.

Description

Technical Field

[0001] The present invention relates to a music generation and visualization apparatus and method using MIDI control and 7 Color Materials music theory (7CM theory: brilliance theory, gravitation theory, function theory, tonality display). In particular, it relates to complex chord progression conversion by 7CM tonality selection and interactive control of brilliance sense, atmosphere, and chord progression sense according to the user's trends and external information.

Background Art

[0002] As music develops and diversifies, traditional music theory sometimes leads to inconsistencies in its interpretation of tonality, and there are times when a theoretical framework for understanding tonality and function is needed. In particular, while traditional music theory, which is based on major / minor keys, has contributed greatly to the development of music, there is room for improvement in its ability to flexibly respond to new tonal structures and expressive techniques. For example, in traditional music theory, chords are classified into tonic, dominant, and subdominant functions. This classification definition is problematic because, for example, IIIm in C major (III represents the Roman numeral 3) is defined as tonic, but depending on the context, it may be considered dominant, and it can be interpreted as tonic or dominant depending on the subsequent context, making a unambiguous definition of function difficult. In another example, when CM7 and Em progress to F, they have the same function, and when Bm7♭5 and G9 progress to C, they have the same function, the explanation given is that "the root note is omitted, so the function is equivalent." The reasoning is that if you omit the "C" in CM9 (C-E-G-B-D), you get Em7 (E-G-B-D). However, this reasoning leads to a logical difficulty where, by repeating C (C-E-G) = CM9 (C-E-G-B-D) = Em7 (E-G-B-D) = G (G-B-D) = G9 (G-B-D-F-A) = Bm7♭5 (B-D-F-A) = Dm (D-F-A)..., all chords within a key that should inherently have different functions become functionally equivalent, such as C=Dm=Em=F=G=Am=Bm♭5. Therefore, a systematic reasoning or definition that avoids this is needed. Also, non-reference 7 is a book published by the press of Berklee College of Music in the United States, which is described as the world's best music university. In this book, it states that "Vm7 cannot be called a dominant by definition because it lacks a tritone. However, even without a leading tone, it can function as a substitute for a true dominant 7th chord depending on the context," suggesting that the definition of function in music theory can be context-dependent. Such fluctuations in the uniqueness of definitions, while offering flexibility in practice, also present challenges in theoretical construction. These are just examples, but given the current challenges in theoretical organization, directly applying the principles and rationale of a theory to technology comes with certain limitations. For example, in the former case, the principle that IIIm is a tonic is not always applicable, and there are exceptions, so we are forced to rely on knowledge from successful examples where it could be used as a tonic.Furthermore, in the latter example, since omitting the root consonant is not a principle that can always be used, there is a challenge in that only a limited number of techniques that depend on successful examples can be offered, such as applying it only to successful cases.

[0003] If axioms that can uniformly logically explain the problems inherent in conventional music theory are discovered, these problems can be resolved by constructing a technique that directly utilizes that axiomatic system as a principle. There is a music theory called 7 Color Materials that was constructed with the aim of organizing an axiomatic system that solves these problems and describing musical phenomena without contradiction. Hereafter, we will refer to it as the 7CM theory.

[0004] While there are existing technologies for interactively changing music in response to user actions such as game state changes and external information such as environmental changes, such as dynamic music generation through playback order control of music sequences, dynamic music generation that dynamically selects playback targets from multiple music sequences or controls the volume of each sequence, and dynamic implementation of pitch changes or timbre changes for the entire song or specific instruments, technologies for directly changing the tonality of a song in real time as a musical element, or technologies for real-time chord candidate consideration and modification so that the progression is equivalent to the original progression, remain unestablished. This poses a challenge in deepening the immersion of the musical experience, the connection with the experiential events, and the sense of unity in response to user actions and external information. Patent document 1, JP 2020-127714, discloses an interactive music generation technology, but this is an editing method that links the climactic parts of an existing song with the climactic parts of a video game highlight, and does not change the tonal structure of the song. While Patent Document 2, Japanese Patent Application Publication No. 2002-117069, also discloses an interactive music generation technology, it generates music from items in a pre-registered database and does not change the tonal structure of the music being played.

[0005] Regarding the alteration of tonal structure, while MIDI technology provides channel-level pitch processing techniques, these techniques alter the pitch of all musical notes within a channel. Therefore, techniques for processing only specific intervals of pitch for tonal changes have not been established. Patent Document 3, JP 2025-006344, discloses a pitch control technique that is limited to guitars and specific to guitar strings. However, it cannot be applied to general instruments or tonal management in played music, nor to interval-level pitch control for tonality. Patent Document 4, JP 10-049150, describes a technique for processing pitch bend independently for multiple chorus melody data. However, this technique controls within a defined pitch range, and therefore cannot achieve management and control at the interval level, which is the unit that constitutes tonality.

[0006] Furthermore, while existing music production applications and support tools offer music control functions based on conventional music theory knowledge, such as chord notation, chord suggestion / selection, scale notation, scale suggestion / selection, and melody suggestion / selection, the format of suggesting candidates based on conventional music theory and knowledge bases makes it difficult to grasp the complex tonal relationships described by 7CM theory, construct melodies and chords through effective tonal design, understand the musical role of chord progressions formed by the framework of chord progressions (not a description of roles based on the contradictory functional system of tonic, subdominant, dominant, etc., which were cited as an example of the problems of conventional theory, but a description of roles based on a different logic that resolves this problem), and gain an overview of the musical context within other tonalities for intuitive chord progression construction and grasping opportunities for chord borrowing and modulation. As a result, the creativity of composers and sound designers has not been fully unleashed.

[0007] In particular, when it comes to the use of complex chords, there are only a few means to either utilize existing knowledge (such as analysis of existing musical works and knowledge accumulated over many years of music history) or, when creating new chords, discover them by chance while playing an instrument. It has been difficult to create music that is unique and free based on emotions, to construct complex chords by users who do not have extensive experience playing instruments, and to create interactive and situationally immersive music that responds to external information (such as the game situation, user movements, and brightness of the user's point-of-view video). For example, Patent Document 5, JP 2019-507389, describes a platform technology that makes it easy to create music without knowledge of music theory, but it is a technology that synthesizes and creates music based on existing music content, and has the challenge of having to rely on prepared musical success knowledge (including large things such as existing knowledge, know-how organized in conventional music theory such as cadences, and collections of chord progression patterns of popular songs, as well as small things such as the judgments that the creators of the existing music content themselves deemed good at the time of production).

[0008] Furthermore, while existing music production applications and support tools utilize music control functions that allow for the reuse of melodies and chord progressions by copying and pasting at the level of actual pitches or chord symbols, reusing only the atmosphere of a tonality, or only the context or structure of a melody or chord progression, remains difficult, posing a challenge to the efficiency of composers and sound designers. For example, if one wants to reuse only the atmosphere of a chord progression like F→G7♭9→C, such as Dm7♭5→C, or reuse the context of the same chord progression, such as G / F→G→C, there is no technology to reuse at the level of atmosphere or context. In the case of MIDI information, it is necessary to reuse at the level of actual pitches or chords and then make modifications. [Prior art documents] [Patent Documents]

[0009] [Patent Document 1] Japanese Patent Publication No. 2020-127714 (Method and system for generating audiovisual content from video game footage) [Patent Document 2] Japanese Patent Publication No. 2002-117069 (Sequence generation method) [Patent Document 3] Japanese Patent Publication No. 2025-006344 (Control device, musical sound generation method, and musical sound generation program) [Patent Document 4] Japanese Patent Publication No. 10-049150 (Music sound control device, karaoke device, music information supply and playback method, music information supply device, and music playback device) [Patent Document 5] Japanese Patent Publication No. 2019-507389 (Apparatus, system, and method for generating music) [Non-patent literature]

[0010] [Non-Patent Document 1] 7CM Music Theory Explanation Site (https: / / www.7cm.space / ) [Non-Patent Document 2] Wikipedia (strong progress) https: / / ja.wikipedia.org / wiki / %E5%BC%B7%E9%80%B2%E8%A1%8C [Non-Patent Document 3] Tokyo University of the Arts, Center for Art Information: Distribution location for Interactive Music course materials (reference for acquiring motion and color information via Puredata through a webcam) (https: / / puredatalesson.blogspot.com / 2014 / 06 / 10web.html) [Non-Patent Document 4] MobMuPlat(https: / / www.danieliglesia.com / mobmuplat / ) [Non-Patent Document 5] Music control system and method based on 7CM theory (implementation sample) (https: / / www.7cm.space / patent / ) [Non-Patent Document 6] Audiokinetic's Wwise learning content: Learn Wwise (https: / / www.audiokinetic.com / ja / learning / learn-wwise) [Non-Patent Document 7] The Berklee Book Jazz Harmony / Joe Mulholland, Tom Hojnacki (ISBN:978-0-87639-142-6) [Non-Patent Document 8] The Complete Guide to Chord Theory / Hibiki Shimizu (ISBN: 978-4845632367) [Overview of the project] [Problems that the invention aims to solve]

[0011] To explain the assignment, I will first provide an overview of the 7CM theory described in Non-Reference 1. The 7CM theory is broadly divided into "Attraction Theory / Functional Theory" and "Iridescence Theory," and I will explain the overview and applications of the 7CM theory, including the following concepts which are commonly necessary: ​​"7CM," "Single Tonality," "The Concept of Tonality (Overlapping Single Tonalities)," and "Lingering Scent of Sound"; the concepts of Attraction Theory / Functional Theory, such as "Functional Tone," "Attraction (D Attraction, L Attraction, T Attraction)," "Bundle of Tones / Considered Single Tones," "Magnitude of Attraction," "Connection," "Stagnation," "Function and its Validity," and "Suspended Attraction"; and the concepts of Iridescence Theory, such as "Iridescence of 7CM (Lower Key Column, Upper Key Column, Lower Saturation, Main Saturation, Main Brightness, Main Luminance, Upper Luminance)," "Mode System 7CM," and "7CM Multiplication." I will summarize and present the information in a table after the explanation.

[0012] First, let me explain the seven-tone axiom, which is the fundamental concept of this theory. The 7CM theory has the following axioms: There are seven types of musical roles within a tonality, and musical notes are mapped to one of these (the seven-tone axiom). These seven types of roles are called interval classes, and there are seven types, from the 1st to the 7th. For example, in C major, C and C# are mapped to the "1st interval class," and although G# and A♭ refer to the same key, they are classified as the "5th interval class" and the "6th interval class," respectively, and are distinguished as different things. This is a concept of classes (types) that omits ornamentation (augmented, diminished, minor, major) from the intervals as understood in conventional music theory.

[0013] Next, I will explain the unit that captures the state of musical space, which is the fundamental concept of this theory. The musical notes within a key are broken down into a combination of (1) interval classes, which form the framework of the musical structure, and (2) pitch for each interval class. Based on the axiom of 7CM theory that the major scale type (the interval of the seven white keys from C to the B above) is the most stable key, (2) is basically considered as the difference from the major scale. As an example, if we describe the augmentation and decrement in semitone units for the Mixolydian mode of C major in 12-tone equal temperament, which is C, D, E, F, G, A, B♭, then the interval classes are 1st: ±0, 2nd: ±0, 3rd: ±0, 4th: ±0, 5th: ±0, 6th: ±0, and 7th: -1.

[0014] Let me explain "7CM". The set of "(2) pitches for each degree class" mentioned above, that is, the structure of seven relative distance relationships from the central note, is called 7CM. This refers to the scale from the central note in conventional music theory. For example, the 7CM CNM is the 7CM NM with C as its central note, and NM is isomorphic to the major scale. In other words, CNM is equivalent to the C major scale = {Do, Re, Mi, Fa, Sol, La, Si}.

[0015] This section explains "single tonality" and "the concept of tonality (where single tonalities overlap)." The following can be recognized in the axiom system of the 7CM theory. The concept of the above-mentioned 7CM is called single tonality, and the tonality of the music space is a superposition of multiple single tonalities. This is called the tonality composition axiom. As an example, the examples of the C major and D minor can be cited. In the 7CM theory, since the major key of the conventional music theory is denoted by the 7CM names of NM or Ion, and the minor key is denoted by Nm or Aeo, the C major is denoted as CNM, and the D minor is denoted as ANm. In the conventional music theory, the cadence in the Am chord in the C major is called a deceptive cadence, but in recent J-POP, etc., there is a current situation where Am is so commonly used to play its cadential role that the expression "deceptive" is not appropriate. In the 7CM theory, this is treated as a state where the single tonality of CNM and the single tonality of ANm overlap, and it is reasonably explained without special treatment of matters on the ANm side, which is a single tonality.

[0016] Explain the concepts of "functional tone" and "gravitation (D gravitation, L gravitation, T gravitation)" that are concepts of the theory of gravitation / function theory. The following can be recognized in the axiom system of the 7CM theory. The musical tone in the low register is called a functional tone, and it has the property of wanting to connect to other musical tones, or has the property of obtaining a psychological sense of satisfaction when actually connected. This is called gravitation. In the conventional music theory, it is a psychological feeling described by a sense of resolution or a sense of progression. Regarding gravitation, the gravitation to the degree class five degrees lower is called D gravitation, and the gravitation to two degrees higher and two degrees lower is called L gravitation. In terms of the conventional music theory, the connection that becomes a strong progression follows the D gravitation connection, and the connection that becomes a sequential progression follows the L gravitation connection. In particular, all tones have gravitation to the degree class 1, which is the central tone (tonal center) of the tonality, and this gravitation is called T gravitation.

[0017] Explain "sound bundle / virtual single tone". The following can be recognized in the axiom system of the 7CM theory. The set of mid- and high-pitched musical sounds other than functional sounds can be regarded as equivalent to the low-pitched musical sounds in terms of gravitational properties when bundled together. This is called a sound bundle, which is essentially a harmony, and a harmony can be regarded as equivalent to a functional sound in terms of gravitational properties. Also, the functional sound that is regarded as equivalent as a result of bundling is also called a single sound. Note that although there are differences in auditory perception changes due to this bundling method, this property is treated separately from the gravitational property. As an example of the bundling method, the chord progressions C→Em→F, CM7→Em7→FM7, and C→E7→F are accompanied by minor differences in auditory perception and differences in brilliance (described later), but since the structure as the context conveyed by the music sounds the same, it means that this sense of progression is regarded as the same in terms of the concept of functional sounds. Based on this basic idea, all chord progressions can be abstractly considered in the world of functional sounds, and conversely, specific harmonies can be constructed by considering the bundling method (for example, specified as the first, third, and fifth degrees) for the progressions designed with functional sounds. In the example case, the progression of functional sounds is simply represented as C→E→F.

[0018] Explain "the magnitude of gravity". The following can be recognized in the axiom system of the 7CM theory. It is recognized that D gravity is stronger than L gravity. When associated with the terms of traditional music theory, since D gravity is the principle of strong progression and L gravity is the principle of sequential progression, this is a prioritization as the strength of gravity in view of the usage, frequency, and auditory perception of these strong progressions and sequential progressions in music. As an example of the basis, the fact that G→C has a stronger sense of resolution than Dm→C, and that G in this case is distinguished from Dm and called a dominant, etc. can be cited. As a reference, it can be mentioned that the first one described in the "Rank of Progressions in Use" on the Wikipedia strong progression page of Non-Patent Document 2 follows D gravity, and the second one follows L gravity.

[0019] Explain "connection" and "stagnation". In 7CM theory, a transition where a functional tone has an attractive force is called a connection, while a transition without an attractive force is called a stagnation. For example, in the chord progression F→G7→Dm, the transition from F to G7 is described as a connection because an attractive force is at work. The transition from G7 to Dm is described as a stagnation because there is no attractive force.

[0020] Let's explain the "function". In 7CM theory, attraction is the axiom, and functions are defined as the roles related to progression within a single key. The 1st degree is considered the T function. The 5th degree, which has both T and D attraction to the 1st degree, is considered the D function. The 2nd and 7th degrees, which have both T and L attraction to the 1st degree, are considered the L function. The remaining 3rd, 4th, and 6th degrees are considered the S function. In terms of properties, the D function has both T and D attraction to the tonic 1st degree, making it a function with a strong attraction to the tonic. Similarly, the L function has both T and L attraction to the 1st degree, making it a function with a strong attraction to the tonic. From the perspective of conventional music theory, T is close to the tonic, D is close to the dominant, S is close to the subdominant, and L is something in between the subdominant and dominant.

[0021] Let's explain the "validity of the function." The axiom that views the tonality of musical space as the superposition of multiple single tonalities (the axiom of tonal composition) largely aligns with the functions of conventional music theory while solving its problems. For example, in contemporary music where major and minor keys coexist, such as the relationship between C major (CNM) and A minor (ANm), conventional music theory and schools of thought have problems with functional uniqueness. For instance, the function of the 7th Bdim is not uniquely determined; for example, connecting it to E7 might be considered a subdominant, or connecting it to C might be considered a dominant substitute. Furthermore, when C progresses to F, it is considered equivalent to Em. As shown in Figure 20, the 7CM theory isolates the tonality consisting of white keys as a state in which multiple single tonalities such as CNM (7CM equivalent to C major), ANm (7CM equivalent to A minor), and Flyd (7CM equivalent to the Lydian mode in F major) are superimposed. When viewed functionally, by considering CNM (T, L, S, S, D, S, L functions in the order of C, D, E, F, G, A, B), ANm (S, S, D, S, L, T, L functions in the order of C, D, E, F, G, A, B), and Flyd (D, S, L, T, L, S, S functions in the order of C, D, E, F, G, A, B) as overlapping, tonality can be rationally broken down and understood. This explains why Bdim has a strong dominant character, as the L function in CNM and the L function in ANm overlap and resolve psychologically, and why C has an aspect that suggests an S function in addition to the T function. This idea that the functional characteristics of a single tonality become stronger depending on the psychological state of which single tonal layer is strongly felt (i.e., which central tone the heart more strongly anticipates), solves the problem of functional uniqueness that has existed in conventional music theory.

[0022] This section explains the concept of "luminosity (undertone column, upper tone column, lower saturation, primary saturation, primary lightness, primary luminance, upper luminance)" which is the basis of luminosity theory. This expresses the atmospheric role of the frequency classes in terms of luminosity. The following axioms are recognized in the 7CM theory: The unison and fifth intervals are the pillars that support tonality. For example, in the Locrian mode (when the tonic is C, the notes are C, D♭, E♭, F, G♭, A♭, B♭), the relationship between the unison and fifth intervals is a diminished fifth in actual pitch, which gives it a sense of disintegration and instability. Thirds are responsible for the brightness of a tonality and are called principal brightness. For example, when a piece of music in C major is changed to C minor and played, the note E-flat may sound darker. The sixth intervals are responsible for the color and vibrancy of a tonality and are called the primary saturation. One example is the A-flat note in a C major piece, particularly the Fm subdominant minor, which has a bright yet sepia-like, somewhat melancholic sound. Seventh intervals carry emotions such as the sharpness of tonality or the intensity of a light source, and are called principal luminosity. Examples include the feeling of a sharply honed atmosphere in darkness, achieved by changing a G to a G# in an A minor piece, or the peaceful feeling of the Mixolydian mode in C major (in this case, a state of reduced luminosity). Thirds, sixths, and sevenths are collectively called the principal splendor. For example, lowering all the principal splendors of a major key by a semitone results in a minor key, which is considered the main element of the tonality. The second degree corresponds to the tonic chroma of the subdominant key, and is also called the sub-chroma because the subdominant key is a closely related key. For example, the note D in C major is the sixth degree in F major and is its tonic chroma. The fourth interval is the tonic of the dominant key, and because the dominant is a closely related key, it is called the superdominant interval in contrast to the subdominant key. For example, the F in C major is the seventh in G major and is its tonic interval. In 7CM theory, these are called "lusters," and the theory related to them is called "luster theory" within 7CM theory. In 12-tone equal temperament, these various lusters are sometimes expressed as semitone differences from the major scale, NM.

[0023] Let me explain the "Mode-style 7CM" model. Of the 7CM modes, those that expand to NM are called modal 7CM modes. For example, Lyd refers to the Lydian mode in conventional music theory, and the actual notes of C Lyd, with C as its tonic, are {C, D, E, F#, G, A, B}. Lyd, Ion (also known as NM), and Mixo are called major modal 7CM modes, while Dor, Aeo (also known as Nm), and Phhr are called minor modal 7CM modes.

[0024] Let's explain "multiplication by 7cm". In 7CM theory, HM is defined as a variation of NM, and MM and WM are defined as variations of HM. Similarly, Hm, Mm, and Wm are defined for Nm. 7CM theory extends this to modal 7CM, defining LydHM, LydMM, LydWM, MixoHM, MixoMM, and MixoWM as multiplications of major keys Lyd and Mixo with HM, MM, and WM, respectively. For minor keys Dor and Phr, DorHm, DorMm, DorWm, PhrHm, PhrMm, and PhrWm are defined as multiplications with Hm, Mm, and Wm. In this invention, these will be referred to as the H-ification of Lyd, the W-ification of Dor, and so on. The content of 7CM is defined as follows: in major keys, it refers to HM, MM, and WM in keys where the pre-multiplication 7CM has NM; and in minor keys, it refers to Hm, Mm, and Wm in keys where the pre-multiplication 7CM has Nm. For example, CMixoHM is a set of seven notes, and since CMixo = {C, D, E, F, G, A, B♭} = {F, G, A, B♭, C, D, E} = FNM, the content of CMixoHM is the structure {C, D♭, E, F, G, A, B♭} when the central note is C (C) in the inversion of FHM = {F, G, A, B♭, C, D♭, E}.

[0025] This section explains the concept of "lingering sound," which is common to both theories of attraction / function and theories of luminosity. The following is recognized in the axiom system of 7CM theory: The sense of tonality and musical tones are maintained in the mind for a certain period of time. This property is called lingering scent. An example of lingering scent related to brilliance is when playing in Clyd (Lydian mode in C major), where one anticipates or expects F# even when not directly hearing F#. In other words, it is a psychological state where one is no longer surprised by the appearance of F#. Another example of lingering scent related to attraction is when the C in the chord progression C→F is split into C→Em→F or C→(blank)→F, where one recalls the sense of resolution that was originally expected at the C point, leading to F. In this case, it can be said that the lingering scent of the first C progressed to F. The latter lingering scent related to attraction is specifically called retained attraction. Simply put, retained attraction is the attraction to a functional tone two notes ahead, skipping one functional tone. For example, in the chord progression Am→F→G, the anticipation for G at the Am chord is called the "retention attraction," and this concept allows progressions like Am→F→G and Am→Am→G to be viewed as isomorphic within a musical context.

[0026] This section explains the concept behind the 7CM theory, which is deeply related to this invention. This explains the relationship between 7CM and actual musical tones (such as melodies and chords), and what it means musically to change 7CM. Tonality refers to a state in a piece of music where a specific note is the central tone, and other notes are systematically arranged based on that central tone. Melodies and chords exist within that tonality, and therefore are notes within the scale defined by that tonality. Conversely, since the actual sounds heard by melodies and chords form tonality in the mind, it is self-evident that melodies and chords become tonal notes.

[0027] I will explain the concept of application. As mentioned above, melodies and chords form tonality, but the idea behind applying 7CM theory is to design tonality. In other words, by determining the tonality, melodies and chords are transformed into notes that make up that tonality, thereby achieving the intended sense of tonality. For example, consider a situation in C major (CNM) where the melody is "F / G / A / A" and the chord is F (F-A-C), "G / F / E / D" and the chord is G (G-B-D), and "C / B / A / B / C" and the chord is C (C-E-G). These are all composed of notes from CNM = {C, D, E, F, G, A, B}. Now, if we change 7CM to CHm = {C, D, E♭, F, G, A♭, B}, the melody and chords will change to "F / G / A♭ / A♭" and the chord is Fm (F-A♭-C), "G / F / E♭ / D" and the chord is G (G-B-D), and "C / B / A♭ / B / C" and the chord is Cm (C-E♭-G). The listener will feel as if they are listening to the same song but with a different atmosphere.

[0028] Table 1 shows the axioms, Table 2 shows the definitions, and Tables 3 and 4 show the conventional approach in music theory and the approach in 7CM theory. The content is the same as above. [Table 1] [Table 2] [Table 3] [Table 4]

[0029] In translating these 7CM theory axiomatic systems into music production technology, we achieve tonal changes for the entire piece or parts of a piece by implementing tone management and processing specifically for tonal changes, and we solve the problems of conventional technology by enabling the saving and reuse of tonality and functional tone progressions in file formats. From this point forward in this application, frequency classes will be treated without regard to their class, and therefore the term "frequency" will be used.

[0030] Furthermore, by formalizing the theoretical elements that describe phenomena in musical space using the 7CM theory and directly applying the axiomatic system as technology, we can achieve the visualization of tonal states and inter-tonal relationships in 7CM units, visualization of tonal states using luminosity parameters, tonal design by selecting 7CM units, tonal design by manipulating luminosity, melody and chord construction by changing tonality, visualization of the musical role of functional tones which form the backbone of chord progressions, visualization of the attractive forces between adjacent functional tones and two functional tones ahead which represent the contextual structure of music, construction of chord progressions using intuitive functional tone progressions that intertwine functional tones and emotions, visualization of the functional role that chord progressions have in other tones for mastering chord borrowing and modulation opportunities, and an operation method that allows for easy chord construction regardless of keyboard instrument playing experience, thereby solving the problems of conventional technology.

[0031] Furthermore, by specifying parameterized radiance and categorized attraction, the challenges of interactive tonal and chord progression changes are addressed. [Means for solving the problem]

[0032] This invention provides a music control system for managing and controlling tonality and functionality in music production. The music control system of this invention incorporates the management and control of musical characteristics based on the 7CM theory as an essential component, and enables intuitive and efficient music production by adding visualization and UI operation as needed. In particular, it solves the problems of conventional music production through the following means.

[0033] 1. Management of tonality based on 1.7CM theory

[0034] The claims described in Claims 1, 3, 4, 9, 10, and 11 are carried out as follows.

[0035] The system centrally manages various types of information (241) in block units, including 7CM files (1111) stored in the storage device (111) shown in Figures 1 and 3, UI (M12) operation of the 7CM display / selection device shown in Figure 7, MIDI input (202) shown in Figure 2, and key input (401) from a PC keyboard shown in Figure 4, as specified 7CM information, tonal information, information identifying the relative key state, and information about the H / M / W components that make up 7CM. This allows for a clear definition of the tonality of the entire piece or a specific part of it, enabling intuitive handling of complex tonal changes and chord progressions. Furthermore, it enables the realization of changing the tonality of the entire piece or a part of it by selecting an arbitrary or similar tonal 7CM, and enables saving and reusing tonal files, thereby solving the problem.

[0036] In particular, by visually representing adjacent parallel keys, modal 7CM systems, and H / M / W-enhanced 7CM states on the UI (M11) shown in Figure 6, musical relationships can be easily understood, supporting creative music production. This enables the visualization of tonal states and inter-tonal relationships in 7CM units, solving the problem.

[0037] Realization of brilliance based on the 2.7CM theory

[0038] The methods described in claims 1, 2, and 7 are implemented as follows.

[0039] Information about the illumination value specified via the MIDI information PolyTouchIn (201) shown in Figure 2, the UI of the illumination sensitivity specification device (G1-1 in Figure 5), and the UI of the 7CM display / specification device (M12 in Figure 7) is managed in real time. To achieve flexible tonal control, MIDI information output (252) related to notes is output on multiple channels for each degree, but by always synchronizing the illumination value with the output pitch bend state, the atmosphere / tonal space intended by the user is realized. This enables tonal design through illumination manipulation and melody and chord arrangement through tonal changes, solving the problem.

[0040] In particular, the luminosity corresponding to various input information is visualized in real time on the UI of the luminosity specification device (G1) shown in Figure 2 (G1-1 in Figure 5). This enables the visualization of luminosity information in the tunic state and solves the problem.

[0041] 3. Structuring music using functional tonal progressions

[0042] The methods described in claims 1, 5, 6, 9, and 10 are implemented as follows.

[0043] The functional sound progression file (1112) shown in Figure 3, the UI of the functional sound specification (F1) device (F11 in Figure 8), and the functional sounds specified via key input from the PC keyboard shown in Figure 4 are managed as various types of information (241) in block units. This makes it possible to visually grasp and edit the functional progression that forms the backbone of the music. Furthermore, it enables saving and reusing functional sound progressions in a file format, solving the problem.

[0044] The UI (F11) of the functional tone selection device shown in Figure 8 represents the emotions associated with functional tones within a key signature, such as tension and anticipation of progression, as a two-dimensional line graph, enabling chord progression design by visualizing emotional fluctuations. This visualizes the musical role of functional tones, which form the backbone of chord progressions, and realizes the construction of chord progressions using functional tone progressions that intertwine functional tones with emotions, thereby solving the problem.

[0045] Furthermore, by arranging multiple UIs (F11) for the functional tone designation device, and allowing simultaneous reference to multiple monotonals, the functional tone progressions of chord progressions in other monotonals are visualized. This enables the visualization of the functional role that chord progressions play in other tonalities, which is necessary for understanding chord borrowing and modulation opportunities, and supports the user's free chord progression and song concept design.

[0046] Furthermore, by visualizing the attractive force of functional tones on the functional tone progression connection visualization unit (F12) shown in Figure 9, the attractive force between adjacent functional tones and between functional tones two steps ahead, representing the contextual structure of the music, is visualized, thus solving the problem. This enables an intuitive understanding of the musical flow and sense of anticipation, and realizes the creation of more natural and musical chord progressions.

[0047] 4. Flexible sound expression according to the performance mode.

[0048] The methods described in claims 1, 8, and 9 are implemented as follows.

[0049] In response to the MIDI note input (202) in Figure 2, the note conversion output device (N1) calculates the note number and pitch bend using the 7CM, functional sound, and light information stored in various information (241), and then outputs the MIDI (252).

[0050] Within this process, as shown in the example in Figure 14, the system performs a reinterpretation of MIDI input note numbers based on the selected performance mode (Melody mode (N1-31), Bass mode (N1-41), Harmony mode (N1-51), etc.). By simply repeatedly playing the same keys and changing the function sounds and 7CMs, the user can experience various changes in melodies, bass phrases, and chord progressions as MIDI information corresponding to each performance mode is output. This allows the user to focus solely on tonal design and function sound progression design, enabling an easy method of constructing chords regardless of keyboard instrument playing experience, thus solving the problem.

[0051] 5. Intuitive and efficient UI for ease of use

[0052] The claims described in Claim 1, Claim 2, Claim 3, Claim 4, Claim 5, Claim 6, and Claim 11 are implemented as follows.

[0053] For the light intensity specification (G1) in Figure 2, refer to the light intensity change screen operation unit (G1-1) in Figure 5; for the 7CM display / specification (M1) in Figure 2, refer to the 7CM display device (M11) in Figure 6 and the 7CM specification device (M12) in Figure 7; for the function sound specification (F1) in Figure 2, refer to the function sound progression specification unit (F11) in Figure 8 and the function sound progression connection visualization unit (F12) in Figure 9; for the center tone specification (C1) and performance mode specification (P1) in Figure 2, refer to the respective specification UIs (241-12, P1-1) in Figure 11; for the sequence specification (S1) in Figure 2, By providing dedicated UIs for manipulating and confirming musical elements, such as the UIs in Figure 13 (S1-11 to S1-14, S1-21 to S1-24), the converted note visualization UI in Figure 12 for the note conversion output (N1) in Figure 2, and the UIs for reading / writing files related to 7CM and functional sounds (M12-531 to M12-533 in Figure 7, F12-31 to F12-33 in Figure 10), the system simplifies complex parameter settings and improves the efficiency of music production.

[0054] In particular, by providing an operation section that offers visual feedback, users can instantly check the results of their operations, resulting in a more intuitive and smooth music production experience. These features enable the visualization of tonality and the relationships between tonalities in 7CM units, visualization of tonality using glow parameters, tonality design by selecting 7CM units, tonality design by manipulating glow, melody and chord construction by changing tonality, visualization of the musical role of functional tones which form the backbone of chord progressions, chord progression construction using functional tone progressions that intertwine functional tones with emotions, and visualization of the attractive forces between adjacent functional tones and two functional tones ahead that represent the contextual structure of the music, thereby solving the challenges.

[0055] 6. Interactive tone control based on user behavior and external information.

[0056] The methods described in Claims 1, 2, and 12 are implemented as follows.

[0057] The music control method of the present invention includes means for interactively controlling tonality based on 7CM theory using user behavior and external information as input. User behavior and external information include user actions, state, and environmental information, and include, for example, status information obtained from a game engine (e.g., player's HP), color space information detected by a webcam (e.g., user-viewpoint camera footage worn by the user), and smartphone sensor information (e.g., tilt information). Based on this information, for example, if the in-game player's HP is 50% or less, 7CM is changed from "NM" to "Nm", and if it is 20% or less, it is set to "Hm", thereby changing the tonality according to the game situation, such as creating darkness or a sharp sense of tension.

[0058] This will enable interactive changes to the musical tone in response to user behavior and external information, aiming to solve the problem of enhancing the immersive user experience.

[0059] 7. Interactive chord progression control based on user behavior and external information.

[0060] The methods described in claims 1 and 13 are implemented as follows.

[0061] The music control method of the present invention includes means for interactively controlling chord progressions based on 7CM theory using user behavior and external information as input. User behavior and external information include, for example, the game situation (e.g., solving a puzzle), user actions acquired by a webcam (e.g., waving), and smartphone sensor information (e.g., walking speed). The method changes the chord progression to one where the intervals between functional sounds are paused when the game is stalled or the user shown on the webcam is not moving much, or to one where the intervals between functional sounds are connected when the walking speed is above a certain speed.

[0062] This enables interactive chord progression changes in response to user behavior and external information, aiming to solve the problem of enhancing the interconnectedness and sense of unity in the user experience.

[0063] Through these means, the present invention provides powerful tools for music producers to create music more creatively and efficiently, and contributes to expanding the possibilities of musical expression. Table 5 shows the relationship between the problems, the corresponding solutions, and the claims. [Table 5] [Effects of the Invention]

[0064] The music control system of the present invention (Claim 1) enables flexible music generation or editing according to the application without requiring specialized knowledge of music theory, by managing and controlling musical characteristics based on the 7CM theory. In particular, by providing visualization functions and an operable UI that can be added depending on the application (Figure 5: Claim 2, Figure 6: Claim 3, Figure 7: Claim 4, Figure 8: Claim 5, Figure 9: Claim 6), it contributes to effective and efficient music production through intuitive operation without relying on specialized knowledge of music theory or limited existing knowledge. Specifically, the system visualizes the principles of atmosphere inherent in tonality, such as the sense of brilliance, the principle of functionality within a single key, such as functional tones, the attraction between functional tones that is the principle of chord progression, and the principle of the superposition of single tonalities that fluctuates within a key, using a unique UI. By simply making changes through intuitive operation, it becomes easy to construct and design melodies and chord progressions without knowledge of difficult chords or chord progressions.

[0065] The luminosity setting device (Figure 5: Claim 2) enables luminosity value setting, and the corresponding note output from the MIDI note conversion output device (Claim 7) allows for the visualization of luminosity based on the 7CM theory using an intuitive UI (slider), making it possible to generate music with subtle luminosity changes through microtonal expression. For example, in a simple C major piece of music, subtle adjustments to the main luminosity (main brightness: 3 degrees, main saturation: 6 degrees, main luminance: 7 degrees) can be intuitively controlled to make the music slightly darker or brighter.

[0066] The visualization of tonality states realized by the tonality display device (Figure 6: Claim 3) allows for the confirmation of similar tonalities, making it possible to consider 7CM changes such as borrowing chords from similar tonalities, modulating to similar tonalities, and making H / M / W changes on similar tonalities, thereby enabling the tonal design of complex melodies and chords. This allows for the intuitive design of chord borrowing and modulation based on visual information.

[0067] The 7CM selection device (Figure 7: Claim 4) enables real-time 7CM changes, visualization of neighboring keys and modulation (Claim 11), corresponding changes in illumination (Claim 2), and note output from the MIDI note conversion output device (Claim 7), making it possible to compose and arrange music while checking the atmosphere of the 7CM in real time. Furthermore, by changing the 7CM applied to pre-prepared music data, it is possible to dynamically change the atmosphere without compromising the musical context. For example, by changing the 7CM to HM, Aeo, Lyd, etc., in response to an FM7 chord input with a central note of C, it becomes possible to hear complex chord progressions such as FmM7, Fm7, and F♯m7♭5 in real time and check the tonal atmosphere.

[0068] The functional sound visualization setting device (Figure 8: Claim 5) enables real-time functional sound changes, and the corresponding note output from the MIDI note conversion output device (Claim 7) allows for chord progression design while checking in real time chord progression patterns that have a similar musical progression. For example, when there is a chord progression of Am→G7→C in CNM (C major), the functional layer visualizes the progression S→D→T, and it becomes possible to consider equivalent progressions such as F→G7→C while checking the UI.

[0069] The functional sound visualization setting device (Figure 8: Claim 5) displays a two-dimensional line graph of functional sounds corresponding to emotional fluctuations, enabling the design of chord progressions based on emotion.

[0070] The functional sound modification realized by the progression expectation visualization device (Figure 9: Claim 6) and the functional sound visualization setting device (Figure 8: Claim 5) makes it possible to design chord progressions that are aware of the attractive state between functional sounds that are not confined to a single tonality. This makes it possible to arrange chord progressions that maintain the context that the chord progression had within a single tonality while changing the sense of connection as a chord progression to be stronger. For example, Em→G→C is a chord progression that functionally is S→D→T, with Em→G being a stagnation. Even with the same S→D→T, by re-selecting the S function from F or Am, an L attractive force is created between F→G or Am→G, making the state between chords more connected and resulting in a stronger chord progression.

[0071] The functional sound visualization setting device (Figure 8: Claim 5) enables the visualization of chord progressions in other keys, allowing for consideration of shifting towards a different key. The 7CM selection device (Figure 7: Claim 4) allows for the selection of 7CM chords close to that key, enabling various considerations, including the borrowing of chords (double dominant, secondary dominant) as defined in conventional music theory. Furthermore, the visualization of the key allows for further arrangement considerations that encompass conventional concepts, such as H / M / W changes within that key and the design of borrowing states other than dominant chords. Even complex chords can be actively and intentionally adopted visually and intuitively by the user without relying on deep musical knowledge or experience.

[0072] The performance mode designation device (Figure 14: Claim 8) provides a means for easily creating music for each part of the song's structure, making it possible for even beginners to easily create music regardless of the user's performance skills. For example, in music production and performance, basslines are often phrased around the chord root, chords are often constructed by stacking thirds (playing the unison, third, fifth, or seventh simultaneously) around the chord root, and melodies are often phrased without considering chords. By reinterpreting the input MIDI notes according to the nature of each role, phrases, chords, and melodies can be easily constructed. For example, with basslines, the same phrase "Do-Re-Mi" remains, but by changing the functional note, it automatically converts to "Do-Re-Mi" when the functional note is C, and to "La-Ti-Do" when the functional note is A. Similarly, with chords, the keyboard input of "Do-Re-Mi" is reinterpreted to output the 1st, 3rd, and 5th degrees of the functional note, so when the functional note is C, it produces "Do-Mi-So," and when the functional note is changed to A, it produces "La-Do-Mi." This allows for easy conversion of chords by simply changing the functional note while maintaining a simple keyboard input format, making it possible to easily explore various phrases and chords without requiring a high level of playing skill.

[0073] By enabling the setting of core tone, function tone, 7CM information, illumination value information, performance mode, and sequence specification information using MIDI note information (Figure 2: Claim 9), various setting values ​​can be changed on the MIDI device that inputs the performance information. Furthermore, it becomes easy to create MIDI tracks for each information unit, such as 7CM information and illumination information, save MIDI files, and reuse them. As an example of the former, for example, by using the performance mode function (Claim 8) to play the same key repeatedly while playing the key that changes the function tone or core tone, it is possible to easily change the note output to the intended key or chord. As an example of the latter, for example, by managing tracks for information on function tone and tracks for information on 7CM separately, when creating different atmospheric versions of a song with similar chord progressions for the first and second verses, it becomes possible to flexibly reuse such as duplicating only the tracks other than the 7CM information and changing only the 7CM information to create a different version of the chord progression with only the atmosphere changed.

[0074] By realizing file-level management of musical characteristics (functional sounds, 7CM) decomposed by the 7CM theory (Figure 3: Claim 10), it becomes possible to reuse only the sense of progression or only the atmosphere, compared to the reuse of chord progression patterns that have been used in conventional music production.

[0075] An integrated system satisfying claims 1 to 11 makes it possible to generate the framework of chord progressions through functional sound design based on simple input, modify the framework of chord progressions taking into account the increase or decrease in the sense of progression through gravitational visualization, change the sense of tonality through 7CM changes, and provide subtle changes in auditory perception through changes in the sense of brilliance.

[0076] According to the music control method described in claim 12, by interactively changing the tonality based on user behavior and external information, it becomes possible to change the tonality according to various situations, such as the game situation, the sense of color in the user's view, and the user's movements while walking, thereby providing an immersive and personalized music experience.

[0077] According to the music control method described in claim 13, by interactively changing the chord progression based on user behavior and external information, it becomes possible to change the chord progression in response to, for example, changes in game scenes, instructions sent by the user to the screen, walking, etc., thereby realizing a sense of realism, a sense of unity with the experience, and situation-appropriate musical performance.

[0078] As described above, the present invention has the effect of significantly improving operability, versatility, and expressiveness during music production compared to the prior art, and also has the effect of significantly improving the immersion and sense of unity with the user experience based on the 7CM theory as a musical expression of user behavior and external information. [Brief explanation of the drawing]

[0079] [Figure 1] Figure 1 shows a configuration diagram of the music control system. This is a configuration diagram of a first embodiment example implemented via a host application as a plug-in. [Figure 2]Figure 2 shows the processing flow of the device in Figure 1, which processes MIDI input, stores various information, and generates various MIDI output information. [Figure 3] Figure 3 shows the processing flow in which the device in Figure 1 reads and writes external files (7CM files, functional sound progression files) and stores various information. [Figure 4] Figure 4 shows the processing flow of the device in Figure 1, which processes information based on PC keyboard input and stores various types of data. [Figure 5] Figure 5 shows the UI for changing the brightness. There is one UI for each single tonality. According to the axiom of tonal composition, a tonality consists of multiple single tonalities, so there are multiple UIs. To indicate which single tonality that constitutes a tonality takes precedence, the priority tonality state visualization drawing frame (G1-121) highlights the corresponding tonality when it is strong, visualizing the preferred tonality. [Figure 6] Figure 6 shows the visualization UI for 7CM, which is managed in block units. [Figure 7] Figure 7 shows the UI for selecting and configuring 7CM. The settings configured here are immediately applied to the currently selected block. [Figure 8] Figure 8 shows a four-layer graph of functional tone progression managed in block units. By arranging them from top to bottom in the order of D layer (5th), L layer (2nd, 7th), S layer (3rd, 4th, 6th), and T layer (1st), the magnitude of the dynamics of the sense of progression / resolution to the 1st (T layer) is visualized as a two-dimensional difference. By changing the knob for specifying the central tone of the displayed tonality (F11-4), the graph of functional tones in the layer of the tonality with the specified central tone can be viewed in real time. There is one UI for each single tonality. According to the axiom of tonal composition, a tonality consists of multiple single tonalities, so there are multiple UIs. To indicate which single tonality that constitutes a tonality takes precedence, the priority tonality state visualization drawing frame (F11-6) highlights the tonality when it is strong, visualizing the tonality that takes precedence. [Figure 9]Figure 9 shows a visualization graph of the attractive force between functional sounds managed in block units. By arranging the functional sound blocks in a zigzag pattern from left to right, the connection methods between the next functional sound and the functional sound two steps ahead are visualized. The presence or absence of contact boxes allows for a quick visual confirmation of the attractive force between those functional sounds. [Figure 10] Figure 10 shows the UI for inputting / outputting external files for setting the functional sound progression. [Figure 11] Figure 11 shows the UI for displaying the central tone, key, 7CM and relative key status of the selected block, adding / deleting blocks, and changing the playback mode. [Figure 12] Figure 12 shows a UI for monitoring input MIDI note information and output note information after note conversion. [Figure 13] Figure 13 shows the UI for changing the selected block and the displayed block. Changes made here manipulate the sequence number of the block on each device, updating the display in real time. [Figure 14] Figure 14 shows the overall flow of MIDI note information conversion according to the performance mode. [Figure 15] Figure 15 shows the MIDI note information conversion flow when the performance mode is Melody mode. [Figure 16] Figure 16 shows the MIDI note information conversion flow when the performance mode is Bass mode. [Figure 17] Figure 17 shows the MIDI note information conversion flow when the performance mode is Harmony mode. [Figure 18] Figure 18 shows the output flow of pitch bend information corresponding to the luminosity. A separate output channel is used for each degree corresponding to the luminosity, thereby enabling tonal changes through pitch bend. [Figure 19] Figure 19 shows an example of mapping between PC keyboard layouts and various function settings. [Figure 20] Figure 20 shows an example of the state of the axioms (tonality construct axioms) of the 7CM theory that forms the basis of this invention. [Figure 21]Figure 21 shows a conceptual diagram of a second embodiment, which is a different embodiment from Figure 1, that realizes interactive light / tone changes according to the game situation using Wwise. [Figure 22] Figure 22 shows a conceptual diagram of a third embodiment, which is a different embodiment from Figures 1 and 21, that realizes interactive chord progression changes according to the game situation. [Figure 23] Figure 23 shows the flow from the sequence on the MIDI track in the first embodiment to the output to the sound source through this device (VSTi). [Figure 24] Figure 24 shows an example image of sequence creation on a MIDI track in the first embodiment. [Figure 25] Figure 25 shows examples of functional sounds and note conversions during pronunciation before the 7CM change in the first embodiment. [Figure 26] Figure 26 shows an example of note conversion during pronunciation after changing the functional sound and 7CM in the first embodiment. [Figure 27] Figure 27 shows the two-dimensional line graphs and gravity visualization state for keys C, A, and F before the functional sound change in the first embodiment. Here, the state from block 2 to block 9 is displayed. [Figure 28] Figure 28 shows the two-dimensional line graphs and gravity visualization state for keys C, A, and F after functional sound modification in the first embodiment. Here, the states from block 2 to block 9 are displayed. [Figure 29] Figure 29 shows the tonal visualization state of the 7CM change example in the first embodiment. Here, the state from block 2 to block 9 of E1-712 in Figure 26 is displayed. [Figure 30] Figure 30 shows an example of chord progression transformation as an example of visualizing similar tonalities. The original chords are "C→FM7→Bm7♭5→CM7→F", the tonal change is "CNM→CHM→CMM(=CPhrMm)→CPhrMm(=CMM)→CPhr(=FNm)", and the resulting chord transformation is "C→FmM7→B♭7→C7→Fm", an example of being able to modulate to FNm (F minor). [Modes for carrying out the invention]

[0080] As an example of the present invention, a first embodiment, Figure 1 shows a music control system implemented as a plugin that operates on a host application (e.g., a DAW). If necessary, audio output and transmission / reception of MIDI data can be enabled using a computer-connected speaker (113) or MIDI device (114) such as a personal computer, but these are not essential components.

[0081] The storage device (111) shown in Figure 1 saves 7CM files (1111) and functional sound progression files (1112) in text file format.

[0082] Figure 2 shows the general flow when processing based on MIDI information. MIDI information (201) input as PolyTouchIn is processed by the light sensitivity specification device (G1), and the information specifying the light sensitivity is managed into various types of information (241). MIDI information (253) is output as PitchBend information from separate channels designated for each degree (1st to 7th degrees; for example, in C major, C and C# are classified as 1st degrees, E and E♭ as 3rd degrees) that a musical note possesses within a key signature. MIDI information (202) input as NoteIn is processed according to the specified processing for each note number (7CM display / specification device (M1), central tone specification device (C1), function tone specification device (F1), performance mode specification device (P1), sequence specification device (S1), and note conversion output device (N1)). The note conversion output device (N1) calculates the output note according to the values ​​of various information (241) set in each process (M1, C1, F1, P1), and outputs it as MIDI information (252).

[0083] The various pieces of information (241) shown in Figure 2 manage the central note number, parallel key identifier, 7CM name, H / M / W identifier, and functional tone in units called blocks. The parallel key identifier specifies whether the designated 7CM is referenced based on the central note or the parallel key. For example, an HM with C as its central note that is not in the parallel key is treated as a CHM, while an Hm that is in the parallel key and has C as its central note is recognized as AHm={A, B, C, D, E, F, G♯} instead of CHm, with C remaining as the central note. To give a specific example, in the Anpanman March transposed to C major, after a C chord of E, E, C (the part of the lyrics that says "That's right"), an F chord appears with A, G♯, A, B, C, A (the part of the lyrics that says "I'm happy"). Here, the 7CM is AHm={A, B, C, D, E, F, G♯} while remaining in the C major state with C as the central note. In such cases, a combination with the relative key flag is used to express a situation where the tonic is C, but the relative key is Hm in the relative key of A.

[0084] The blocks are numbered sequentially, and various functions refer to this numbering and use this information for calculations. One embodiment of the means for managing these blocks is shown in Figure 13. The current block field (S1-11) and the current block change slider (S1-13) always represent the current block number, and the current block number can be specified by changing these. The current block number can be increased or decreased by one using the current block previous button (S1-12) and the current block next button (S1-14). The display block field (S1-21) and the display block change slider (S1-23) represent the first block number (S1-02) of the 8 blocks displayed in the 7CM display device (M11) in Figure 6, the function sound progression specification unit (F11) in Figure 8, and the function sound progression connection visualization unit (F12) in Figure 9, and the first displayed block number can be specified by changing these. The "Previous Display Block" button (S1-22) and "Next Display Block" button (S1-24) in Figure 13 allow the starting block number to be increased or decreased by one, and each UI redraws various information of the blocks within its display range in real time.

[0085] Furthermore, as shown in Figure 11, the block add button (S1-3) and block delete button (S1-4) allow adding the same block as the current block or deleting the current block. These processes constitute part of the means for "managing the characteristics of music" as described in claim 1.

[0086] Figure 3 illustrates one embodiment of the processing of the 7CM display / designation device (M1) and the central tone designation device (C1) by reading an external file, specifically the process of reading a 7CM file (1111) by a 7CM file reading device (311). The 7CM file (1111) is composed of a format that includes a block number, a central tone note number, a parallel key identifier, a 7CM name, and an H / M / W identifier. One example is "3 96 R Phr H;". The block number is the sequence number managed by this device. In this example, "3" represents this. 96 is the note number of the central tone, which in this example represents C. The parallel tone identifier is "R" when specifying 7CM on the parallel tone side, and "-" otherwise. In this example, "R" indicates this. The 7CM name will be the mode-based 7CM name. In this example, "Phr" indicates this. The H / M / W identifier is "H," "M," and "W" if the mode system 7CM is H-type, M-type, or W-type, respectively, and "N" if it is none of the above. In this example, "H" indicates this. This information expresses, in this example, that "the third block is a 7CM with a central tone of C as APhrHm." This information is stored in various information (241). This process constitutes a means to "arbitrarily specify a 7CM which is a single tonality described in the 7CM theory based on operation, input information, or the contents of an external file" as described in claim 4.

[0087] Figure 3 illustrates one embodiment of saving 7CM information held in various information (241) to an external file: the process of writing (312) to a 7CM file (1111). The format during writing is the same as that during reading, and even if the block sequence numbers are listed with gaps during input, the empty numbers are filled in with NM, which is defined as natural tonality in 7CM theory, and output in ascending order. This process realizes the "management of musical characteristics" of claim 1 and the "means of managing and reusing information of tonal units and functional tone units based on 7CM theory in a file format" of claim 10. For example, if you want to reuse only the atmosphere of a chord progression like F→G7♭9→C (what is called "subdominant minor" in conventional music theory) as Dm7♭5→C, you can save information as HM→NM, thus realizing the reuse of only the atmosphere in different chord progressions or melody playback sections.

[0088] Figure 3 illustrates one embodiment of the processing of a functional sound designation device (F1) using an external file, specifically the process of reading a functional sound progression file (1112) by a functional sound progression file reading device (313). The functional sound progression file (1112) is composed of a format that includes block numbers and functional sounds. Functional sounds are represented by numbers from 1 to 7. For example, "3 5;" indicates that the third block is functional sound 5. This information is stored in various information (241). This process constitutes a means for "sequentially managing a sequence of functional sounds according to operation, input information, or the contents of an external file" as described in claim 5.

[0089] Figure 3 illustrates one embodiment of saving the functional tone progression information held in various information (241) to an external file: the process of writing (314) to a functional tone file (1112). The format during writing is the same as that during reading, and even if the block sequence numbers are listed with gaps during input, the empty numbers are filled in with the preceding functional tone and output in ascending order of block number. This process realizes the "management of musical characteristics" of claim 1 and the "means of managing and reusing information of tonal units and functional tone units based on 7CM theory in file format" of claim 10. For example, if one wants to reuse the context of a chord progression such as F→G7♭9→C, the functional tone progression information 4→5→1 is saved, and then phrasing and other elements are considered to contribute to constructing a development such as G / F→G→C.

[0090] Figure 2 illustrates an overview of one embodiment of specifying the luminosity (251) using MIDIPolyTouchIn information (201) from a host application or PolyTouchIn information (201) from a MIDI device (114 in Figure 1) via the host application. The luminosity to be set (undertone of 1 degree, undersaturation of 2 degrees, main brightness of 3 degrees, upper brightness of 4 degrees, upper tone of 5 degrees, main saturation of 6 degrees, main brightness of 7 degrees) is identified according to the note number of the PolyTouchIn information (201), and the increase or decrease of the luminosity value for that luminosity is determined by the pressure value of the PolyTouchIn information (201). The input MIDIPolyTouchIn information (201) is output directly as MIDIPolyTouchOut information (251) to an external MIDI, the determined luminosity value information is stored as various information (241), the pitch bend value is calculated and output from BendOut to the channel for each degree.

[0091] An example of an embodiment for manipulating and managing luminosity parameters will be explained with reference to Figure 5. The corresponding luminosity is determined according to the note number value (for example, note numbers 0 to 11) of the PolyTouchIn information (201 in Figure 2), the luminosity value is calculated according to the increase or decrease in the pressure value, and this is reflected in real time on the sliders that visualize each luminosity value in Figure 5 (G1-101 to G1-107, G1-111 to G1-114), and stored in various information (241 in Figure 2). For example, note numbers 0 (C on the white keyboard), 2 (D on the white keyboard), 4 (E on the white keyboard), 5 (F on the white keyboard), 7 (G on the white keyboard), 9 (A on the white keyboard), and 11 (B on the white keyboard) could be mapped to the lower key stroke (1st degree), lower saturation (2nd degree), primary brightness (3rd degree), upper brightness (4th degree), upper key stroke (5th degree), primary saturation (6th degree), and primary brightness (7th degree). Other note numbers could then be mapped to three combinations of two primary luminosities and one combination of all three primary luminosities (corresponding to G1-111 to 114). Since the pressure value would range from 0 to 127, one possible calculation method would be to set a pressure value of 0 to -1.0 (representing one semitone lower) and 127 to +1.0 (representing one semitone higher).

[0092] Based on Figure 18, an example of the processing when updating the luminosity value is shown. When each luminosity value is updated, the pitch bend value is output in real time from the output channel defined for each degree corresponding to each luminosity (G1-21). At the same time, it is also stored as various information (241) (G1-3). The pitch bend sensitivity information specific to the output instrument can be specified in advance, and the luminosity can be set for all instruments that support pitch bend. Pitch bend sensitivity defines how many semitones the pitch changes in relation to the maximum pitch bend value of 8191. Some instruments can only change by up to 2 semitones, and in that case, being able to move by a maximum of 2 semitones is sufficient, so the pitch bend sensitivity is set to 2, and when you want to change by 2 semitones, you specify the maximum value in the pitch bend value. As an example of the processing when the luminosity value is changed, if the output channels for lower key stroke (1st degree), lower saturation (2nd degree), main brightness (3rd degree), upper luminance (4th degree), upper key stroke (5th degree), main saturation (6th degree), and main luminance (7th degree) are set to Ch1, 2, 3, 4, 5, 6, and 7 respectively, setting the main brightness (3rd degree) to a value of -1.0 will output information that will result in a semitone lower pitch bend to Ch3 (if pitch bend sensitivity is 2, then -4096; if it is 4, then -2048) in real time, since the luminosity is defined as having main brightness at 3rd degree. This processing realizes the means described in claim 2, which "provides means for manipulating and managing the luminosity parameters of the tonality of a musical space, and dynamically holds and sets luminosity information according to operation, input information, or the contents of an external file," and the means described in claim 7, which "processes input MIDI note information and outputs musical information and pitch bend information from multiple channels."

[0093] Based on Figure 2, an overview of processing branching examples based on MIDI note-in information (202) will be explained. Various processes (M1, C1, F1, P1, S1) based on MIDI note-in information (202) from a host application or from a MIDI device (114 in Figure 1) via the host application are determined to be feasible or not based on the note number of the MIDI note-in information (202) (212 to 216). The input MIDI note-in information (202) is then output to an external MIDI as MIDI note-out information (252), but since it is not an output for sound production, it is output from a channel other than those defined for output for each degree or channels defined for sound production for all degrees (for example, Ch16).

[0094] If the note number is, for example, between 0 and 23, the 7CM display / designation device (M1) processes accordingly and stores the 7CM information for the sequence number of the currently selected block in the various information (241). It is sufficient to have 11 inputs to identify a total of 7 types of mode systems (7CM), 1 type for identifying parallel states, and 3 types for H / M / W conversion. As an example, one possible mapping is to assign note number 12 to Ion, 14 to Dor, and 11 to the parallel key. The input note information can be monitored in real time by the input MIDI note 7CM visualization unit (202-M1) as shown in Figure 12, and the set information is displayed on the 7CM display label (241-2) in Figure 11 and reflected in the 7CM display device (M11) in Figure 6 and the 7CM designation device (M12) in Figure 7. At this time, the 7CM designation device (M12) reflects the luminosity value of the 7CM as luminosity information in real time, so the luminosity value is also set indirectly based on the MIDI note information.

[0095] If the note number is, for example, between 96 and 107, the central tone designator (C1) processes accordingly and stores the central tone information for the sequence number of the currently selected block in the various information (241). One possible mapping is to recognize the note indicated by the note number (12 notes from C note 96 to B note 107) as the central tone. The input note information can be monitored in real time by the input MIDI note central tone visualization unit (202-C1) as shown in Figure 12, and the set information is displayed in the central tone key label (241-11) and central tone note field (241-12) in Figure 11.

[0096] If the note number is, for example, between 108 and 119, the function sound designator (F1) processes accordingly and stores it in the various information (241) as a function sound for the sequence number of the currently selected block. For example, one possible mapping is to assign function tone 1 to 108 (the keyboard note that originally represents the note C) or function tone 2 to 110 (the keyboard note that originally represents the note D). The input note information can be monitored in real time in the input MIDI note function tone visualization unit (202-F1) as shown in Figure 12, and the set information is reflected in the function tone progression specification unit (F11) in Figure 8 and the function tone progression connection visualization unit (F12) in Figure 9.

[0097] If the note number is, for example, between 120 and 124, the performance mode selection device (P1) processes the value accordingly and stores it as performance mode information in various information (241). In this configuration, as an example, there are three modes: Melody mode, Bass mode, and Harmony mode, so input information that identifies four of them, including Off, is sufficient. The performance mode selection button (P1-1) in Figure 11 is also reflected in real time, and the selected performance mode can be confirmed in the UI. As an example, mapping such as 120 to Harmony mode and 122 to Bass mode is possible. The input note information can be monitored in real time by the input MIDI note performance mode visualization unit (202-P1) as shown in Figure 12, and the set information is reflected in the performance mode selection button (P1-1) in Figure 11.

[0098] If the note number is, for example, between 125 and 127, the sequence designation device (S1) processes according to that value, updating the current sequence number and storing it in the various information (241). As a result, the current block is made visible in the information display unit within each device (S1-11, S1-13 in Figure 13, and S1-01, S1-02 in Figures 6, 8, and 9). For example, the Velocity value of note number 125 is recognized as the block sequence number, and when 10 is received, the sequence number is set to 10. The Velocity value of note number 126 is recognized as the difference in block numbers exceeding 127, and when 20 is received, 127 + 20 = 147 is set as the sequence number. The Velocity value of note number 127 is recognized as information to add or subtract one from the current sequence number, for example, based on a value such as 100. If it is less than 100, the current sequence number is subtracted by one, and if it is 100 or more, the current sequence number is added by one. The input note information can be monitored in real time by the input MIDI note sequence operation visualization unit (202-S1) as shown in Figure 12.

[0099] As described above, the process of performing operations on the 7CM display / designation device (M1), center tone designation device (C1), function tone designation device (F1), performance mode designation device (P1), and sequence designation device (S1) based on the input MIDI note information (202) realizes the means described in claim 9 for "setting and managing center tone information, function tone information, 7CM information, luminosity value information, performance mode, sequence designation information, etc., according to the input MIDI note information."

[0100] As shown in Figure 2, if the note number is, for example, between 24 and 95, other than the numbers mapped for the various branches described above, the note conversion output device (N1) processes the various information (241) and outputs it as MIDI note information (252).

[0101] Based on Figure 14, an overview of the processing example of the note conversion output device (N1) when a performance mode is specified will be explained. The input note information (202) is mapped to one of the notes on the white keyboard by performing a white keyboard correction calculation (N1-1), and the mapped note number X1 and the degree calculation result (202a) degree D1 from the white keyboard note C are obtained. In addition, a black keyboard change difference calculation (N1-2) is performed from the input note number X0, and the result is used in the subsequent Bass mode MIDI output processing (N1-42). For example, note number 37 (C♯) is corrected to note number 36 (C), calculated as 1 degree, and the difference for changing black keys is calculated as +1. Depending on the determination of each performance mode (N1-31, N1-41, N1-51), MIDI output processing for each performance mode (N1-32, N1-42, M1-52) is performed. If no performance mode is specified, the output (N1-6) is performed as is, using the input note information (202).

[0102] Based on Figure 15, an embodiment of note calculation (N1-323, N1-327) and note output (N1-324, N1-329) when the performance mode is Melody mode (N1-31) as shown in Figure 14 will be explained. As explained using Figure 18, the pitch bend output corresponding to the luminosity (G1-2) is already output to the channel corresponding to each degree when the luminosity value is changed, and the pitch bend state for each degree is consistent with the luminosity information (241) held internally. When a note is output, the pitch bend is applied and the luminosity is expressed. However, the pitch bend related to the luminosity is not applied to the channel for the overall degree (N1-328).

[0103] First, the note number X2, which is the actual pitch according to the major scale, is calculated using the central tone (information within various information (241)) and degree D1 (N1-323). Then, a process is performed to output the channel for each degree (N1-324), and MIDI note information is output from multiple channels for each degree. As an example, let's explain the input of information for note number 51 (E♭) when the central tone is G. Note number 51 (E♭) is converted to note number 52 (E) in the white key correction calculation process (N1-1) in Figure 14, and since this is a third from the note C, we obtain the information D1 = third. In the note calculation process (N1-323) in Figure 15, the G note (note number 55), which is higher and closest to note number 52, is used as the reference, and the B note (note number 59), which is a third above on the major scale, is calculated as X2. This note number 59 (B note) is output from the channel for the third (=D1), but if the principal brightness is -1.0, the pitch bend value of the channel for the third will be lowered by a semitone, and it will be pronounced as a B♭ note. If the principal brightness is +1.0, it will be pronounced as E♯ (equal to F in terms of pitch). This processing method expresses the degree of brightness by pitch bend.

[0104] Separately, a method for representing the approximation of the luminosity using note numbers will be explained. As shown in Figure 15, by performing a process (N1-327) in which the luminosity value set for each degree is rounded to an integer and added to the note number, a note number that achieves an approximation of the luminosity without using pitch bend is calculated and output from the channel for all degrees (N1-328). For example, when the central tone is G and the main brightness of the radiance is -0.7, input note number 52 (E note) becomes D1=3rd through the note calculation process (N1-323) in Figure 15, and note number X2 is obtained as B note (note number 59), which is 3rd from G. Next, the radiance value rounding process (N1-327) rounds -0.7 down to note number 58 (B♭ note), which is -1.0 semitones lower, and the note output process (N1-329) outputs it from the channel for all degrees (N1-328). Pitch bending to a channel affects the pitch of all notes output to that channel, so this processing method approximates the radiance by adding or subtracting the note number itself without performing pitch bending, and is suitable for applications that change 7CM or radiance in semitone increments and convert notes, such as mode changes. If the input MIDI information includes pitch bend information, it is output as is.

[0105] Based on Figure 16, we will explain the note calculation (N1-422, N1-423, N1-427) and note output (N1-424, N1-429) when the performance mode is set to Bass mode (N1-41) in Figure 14. As explained using Figure 18, the pitch bend output corresponding to the luminosity (G1-2) is already output to the channel corresponding to each degree when the luminosity is changed, and the pitch bend state for each degree is consistent with the internally held luminosity information (241). When a note is output, the pitch bend is applied and the luminosity is expressed. However, the pitch bend related to the luminosity is not applied to the channel for the overall degree (N1-428).

[0106] First, the degree D1 is considered to be the degree on the functional tone of the current block, which is stored in the various information (241), and the degree calculation from the central tone (N1-422) is performed. Since musical degrees are counted with an offset of 1, not 0, care must be taken during the calculation. For example, 1 degree above a 2nd is a 2nd, 3 degrees above a 4th is a 6th, and 5 degrees above a 5th is a 2nd. The processing after the calculation of degree D2 (N1-423, N1-424) is calculated in the same way as the calculation (N1-323) and output (N1-324) of the Melody mode shown in Figure 15, but for the Bass mode, if the functional tone is 5 or more, it is converted to a note one octave lower, and special processing is performed to obtain X2 by adding black key difference information (N1-21). In the former case, for example, if the functional tones are a 7th and a 1st, they are adjacent to each other, but a simple calculation would result in the furthest pitch difference. Therefore, the arrangement 1, 2, 3, 4, 5, 6, 7 is a consideration to shift the notes closer to the tonal interval (1st degree), such as 5, 6, 7, 1, 2, 3, 4. The latter also allows for flexible handling of passing notes and other notes that are outside the tonality. As an example, let's explain the input of information for note number 51 (E♭) when the tonic is G and the functional tone is 4. Note number 51 (E♭) is converted to X1 = note number 52 (E) in the white key correction calculation process (N1-1) in Figure 14, and since this is a third from the note C, we obtain the information D1 = a third. Based on the information of the fourth from the tonic to the functional tone and the third above the functional tone, calculated from the tonic in the degree calculation from the tonic (N1-422) in Figure 16, we obtain the result that the degree from the tonic is D2 = a sixth. The note conversion process in Figure 16 (N1-423) uses the G note (note number 55), which is higher and closest to note number 52, as a reference, and calculates the E note (note number 64), which is a sixth above on the major scale. However, since the input note number 51 is one less than the note number 52 after white key correction, a correction of -1 is applied as black key difference information (N1-21), and the final calculation result is X2 = 64 - 1 = note number 63 (E♭ note). The calculated note number here is output (N1-424) from the channel for each degree (G1-21). This processing method expresses the degree of brightness through pitch bending.

[0107] Separately, we will explain a method that approximates the degree of luminosity using note numbers, similar to the explanation of Melody mode in Figure 15. After the note calculation process corresponding to the frequency in Figure 16 (N1-423), the difference with rounded luminosity values ​​is added to the calculated note number X2 and output (N1-427, N1-429), which is the same as the processing of Melody mode in Figure 15 (N1-327, N1-329). As an example, let's consider the case where the central tone is G, the functional tone is 4, and the saturation of the 6th degree is -0.7, and input note number 52 (E note) is input. Note number 52 (E note) is a third above the functional tone, so D1=3, and it is calculated to be a sixth above the central tone G (note number 55) (X2=note number 64), so D2=6 is obtained. By adding -1.0, which is rounded down from the saturation value of the 6th degree, it is converted to X3=note number 63 (E♭ note), and output from the channel for all degrees (N1-428).

[0108] Based on Figure 17, we will explain the note calculation (N1-521, N1-522, N1-523, N1-527) and note output (N1-524, N1-529) when the performance mode is Harmony mode (N1-51) in Figure 14. As explained using Figure 18, the pitch bend output corresponding to the luminosity (G1-2) is already output to the channel corresponding to each degree when the luminosity is changed, and the pitch bend state for each degree is consistent with the internally held luminosity information (241). When a note is output, the pitch bend is applied and the luminosity is expressed. However, the pitch bend related to the luminosity is not applied to the channel for the overall degree (N1-528).

[0109] First, the white keys are reinterpreted for harmony (N1-521). For example, the 1st, 2nd, 3rd, 4th, 5th, 6th, and 7th degrees are reinterpreted as 1st, 3rd, 5th, 7th, 2nd, 4th, and 6th degrees, and these degrees are designated as the functional tone degree D2. This is because chords usually take the form of stacked thirds (such as the 1st, 3rd, and 5th degrees, or adding a 7th degree), and this is a consideration to make it easier to construct chords of stacked thirds by pressing consecutive keys such as the white keys (Do, Re, Mi, etc.). Based on degree D2, the degree from the tonic is calculated (N1-522), which is the same as the processing in Bass mode (N1-422) in Figure 16. The processing after calculating degree D3 (N1-523, N1-524) is the same as the calculation (N1-323) and output (N1-324) in Melody mode shown in Figure 16. Note that while correction using the difference in black keys was performed in Bass mode, it is not performed in Harmony mode. In other words, it's similar to Melody mode, but not the same as Bass mode. As an example of these processes, we will explain the input of note numbers 48 (C), 50 (D), and 52 (E) when the tonic is G and the functional tone is 4. These are calculated as 1st, 2nd, and 3rd degrees by the white key correction calculation process (N1-1) in Figure 14, and then converted to 1st, 3rd, and 5th degrees through the degree conversion process (N1-521) in Figure 17. In the degree calculation from the tonic (N1-522) in Figure 17, these are converted to 4th, 6th, and 2nd degrees, and output as note numbers 60 (C), 64 (E), and 67 (G), respectively, from the channels for the 4th, 6th, and 2nd degrees, i.e., a C chord (C-E-G). To put it simply, the calculation is that the 4th degree of the tonic G is C, and the 1st, 2nd, and 3rd notes stacked in a 3rd interval from C become a C chord (C-E-G). This means that information entered using the white keys (Do, Re, Mi) is always output in the form of a triad (a three-note chord of 1st, 3rd, and 5th degrees) based on the functional tone, allowing for easy chord construction without playing the black keys. The calculated note numbers are output (N1-524) from channels (G1-21) for each degree. This processing method is a way to express pitch bend and nuance.

[0110] Separately, we will explain a method that approximates the degree of luminosity using note numbers, similar to the explanation of Melody mode in Figure 15. After the note calculation process corresponding to the frequency in Figure 17 (N1-523), the rounded value of the luminosity is added to the calculated note number X2 and output (N1-527, N1-529), which is the same as the processing of Melody mode in Figure 15 (N1-327, N1-329). As an example, let's explain the input of note numbers 48 (C), 50 (D), and 52 (E) when the tonic tone is G, the functional tone is 4, and the saturation of the luminosity is -0.7. These input notes are calculated as information for the 4th, 6th, and 2nd degrees, respectively, yielding note numbers 60 (C), 64 (E), and 67 (G). Since the saturation of the 6th degree is -0.7, a rounded value of -1.0 is added to the 6th degree note (in this case, note number 64, the E note), and finally, the Cm chords of note numbers 60 (C), 63 (E♭), and 67 (G) are output from the channel for all degrees (N1-528).

[0111] As shown in Figure 2, the process (N1) that outputs note conversion in real time based on the input MIDI note information (202), specified center tone, function tone, 7CM, and performance mode specification information as described above realizes the "means for processing input MIDI note information and outputting music information and pitch bend information from multiple channels" described in claim 7 and the "means for processing input MIDI note information and easily creating music for each of the various musical components such as melody, harmony, and bass" described in claim 8.

[0112] Figure 4 illustrates one embodiment of the processing of the 7CM display / designation device (M1), center tone designation device (C1), function tone designation device (F1), playback mode designation device (P1), and sequence designation device (S1) based on key input from a PC keyboard (401). Various processes are performed based on the key input value. As an example of the mapping, as shown in Figure 19, one embodiment of the mapping between key input values ​​and each process is explained, such as the 7CM mode designation button (K1-m1), the 7CM H / M / W designation button (K1-m2), the center tone designation button (K1-c), the function tone designation button (K1-f), the playback mode designation button (K1-p), the sequence operation designation button (K1-s), and the block add / delete operation button (K1-b).

[0113] As shown in Figure 19, if the input keys are, for example, "-", "^", "¥", "p", "@", "[", "l", ";", ":", ",", ".", " / " (K1-m1), the 7CM display / designation device (M1) processes according to the value as shown in Figure 4, and the determined 7CM information is stored in various information (241) as 7CM information for the current sequence number. As in the example in Figure 19, these 12 types of keys can be mapped to the on / off status of 12 7CM (mode system) designation buttons (M12-511) from Phr to Lyd, which are arranged in 3 columns and 4 rows in the center of Figure 7. When an input key is pressed, the UI in Figure 7 toggles the on / off status of the checkbox (M12-511) that enables the valid mode system 7CM.

[0114] As shown in Figure 19, if the input keys are, for example, "r", "e", or "w" (K1-m2), the 7CM display / designation device (M1) processes according to the value as shown in Figure 4, and the determined 7CM information is stored in various information (241) as 7CM information for the current sequence number. In addition, the illumination information of the 7CM is managed in various information (241) in accordance with the determination of the 7CM. One possibility is to map these three types of keys to the row where the currently valid 7CM in the block in Figure 7 is located, interpreting them as H / M / W conversion, respectively. At this time, in the UI of Figure 7, the on / off switch of the 7CM (H / M / W type) designation button (M12-512) for the row where the currently valid mode type 7CM exists is toggled. For example, when Lyd in the second row of M12-512 in Figure 7 is on, pressing "w" on the keyboard turns on WM in the second row of M12-512, making it LydWM.

[0115] As shown in Figure 19, if the input keys are, for example, "←", "→", "↑", and "↓" (K1-s), a sequence specification process (S1) is performed according to the value, as shown in Figure 4. For example, depending on whether it is "→" or "←", the current sequence number may be increased or decreased in the same way as when the "Previous Current Block" button (S1-12) or "Next Current Block" button (S1-14) of the UI in Figure 13 is pressed. Depending on whether it is "↓" or "↑", the current UI display block header may be increased or decreased in the same way as when the "Previous Display Block" button (S1-22) or "Next Display Block" button (S1-24) of the UI in Figure 13 is pressed.

[0116] As shown in Figure 19, if the input keys are, for example, "z", "s", "x", "d", "c", "v", "g", "b", "h", "n", "j", "m" (K1-c), the central tone designator (C1) processes according to the value as shown in Figure 4, and the determined central tone information is stored in the various information (241) as central tone information for the current sequence number. As in the example in Figure 19, these 12 types of key arrangements can be mapped to a keyboard, such as "do, do#, re...".

[0117] As shown in Figure 19, if the input keys are, for example, "1", "2", "3", "4", "5", "6", and "7" (K1-f), the function sound designator (F1) processes according to the value as shown in Figure 4, and the determined function sound information is stored in the various information (241) as function sound information for the current sequence number. For example, if 1 is pressed, the current function sound is set to 1, and mapping matching these seven types of keys with function sound numbers is conceivable.

[0118] As shown in Figure 19, if the input keys are, for example, "y", "u", "i", and "o" (K1-p), the playback mode designation device (P1) processes according to these four types of keys as shown in Figure 4, and this playback mode designation information is stored in various information (241). A mapping is possible that associates four conditions, including no conversion for "y" and three types of playback modes such as Melody mode on for "u", with four types of keys.

[0119] As shown in Figure 19, if the input key is, for example, "Insert," "Delete," or "Backspace" (K1-b), the process of adding a new block after the current block or deleting the currently selected block will be performed, just as when the block add button (S1-3) or block delete button (S1-4) in Figure 11 is pressed.

[0120] As shown in Figure 4, the process (M1, C1, F1, P1, S1) that sets various real-time information (241) based on the key input information (401) from the PC keyboard entered as described above realizes the "management of musical characteristics" of claim 1, the "dynamic retention and setting of luminosity information according to operation, input information, or content read from an external file" of claim 2, the "arbitrary specification of 7CM, which is a single tonality described in 7CM theory, based on operation, input information, or content read from an external file" of claim 4, and the "sequential management of a sequence of functional sounds according to operation, input information, or content read from an external file" of claim 5.

[0121] Figure 5 illustrates one embodiment of the processing of the light intensity specification device (G1) in Figure 2 using a UI, demonstrating the implementation of an intuitive operation method using the light intensity change screen operation unit (G1-1). It is equipped with sliders (G1-101 to G1-107) that display the value of each light and allow it to be changed with mouse operation, and the light values ​​are changed in real time and output as pitch bend values ​​to MIDI channels defined for each degree. Each light value can be changed in real time using increase / decrease buttons (G1-108, G1-109). The light value display number box (G1-110) provided for each light displays the specific light value, and the light value can be changed directly by dragging and inputting. Furthermore, for the two-combination glow of primary luminance and primary saturation, primary saturation and primary brightness, and primary brightness and primary luminance, as well as all three-combination glows, the average value of each glow combination is displayed, and the sliders (G1-111 to G1-114) for changing them simultaneously are also displayed, and the value changed by operating the sliders is reflected in all corresponding glow values. Specifically, for the primary glow and primary brightness sliders, the value of (primary glow value + primary brightness value) / 2 is displayed, and when this value is changed, that value is applied to both the primary glow value and the primary brightness value.

[0122] The light intensity change screen operation unit (G1-1) shown in Figure 5 is equipped with two or more types, one for the dominant key and one for the parallel key, allowing for the visualization and manipulation of light intensity corresponding to the intervals within each single key. For example, the dominant brightness for the dominant key (3rd degree; in C major, E is the 3rd from C) is displayed as the upper key pillar for the parallel key (5th degree; in A minor, which is the parallel key of C major, E is the 5th from A). The dominant key and the parallel key are managed using a parallel key flag to indicate which key the sound is leaning towards. The priority key state visualization drawing frame (G1-121) is highlighted according to this flag, making it clear which of the two existing light intensity change screen operation units (G1-1) should be focused on. For example, when 7CM is in the relative key mode Aeo with a central tone of C, the priority tonality state visualization drawing frame (G1-121) on the relative key's light sensitivity change screen operation section (G1-1) is highlighted, while the same drawing frame (G1-121) on the key side is not highlighted.

[0123] The UI-based specification of the luminosity (G1) described above realizes the means of "dynamically holding and setting luminosity information according to the operation, input information, or contents of an external file" as described in claim 2.

[0124] Figure 6 illustrates one embodiment of 7CM display using a UI, specifically the implementation of a display method using a 7CM display device (M11).

[0125] First, let's explain an example of how adjacent relative keys are represented. The so-called major key NM (synonymous with Ion, written as Ion in notation) easily transitions to the so-called relative minor key Nm (synonymous with Aeo, written as Aeo in notation), and is usually considered simultaneously. One example is the relationship between C major and A minor, such as in a piece of music composed only of white keys. An example of this key is the third from the left in Figure 6. In the second row from the top of the four-row block, the base 7CM drawing section of the tonic major key (M11-20) displays the box for Ion, and the third row from the top, the base 7CM drawing section of the relative minor key (M11-30) displays the box for Aeo adjacent to it, visualizing the closeness between single keys.

[0126] Next, let's explain an example of how to represent the 7CM mode. Each 7CM mode has the property of having a natural tonality (NM or Nm) as its base. For example, the Lydian mode of C major has a high affinity and transitions easily to C major itself. In 7CM theory, this is represented as a fractional notation such as Clyd / NM, with Lyd superimposed on NM. This state is represented by superimposing a rectangular 7CM mode drawing section (M11-21) that does not completely cover the Ion box of the base 7CM drawing section (M11-20) of the key major, which is the second row from the top of the block. The example of the Lydian mode mentioned earlier is the fifth from the left in Figure 6. The Lyd mode 7CM (M11-21) of the key major is displayed superimposed on the Ion base 7CM (M11-20) of the key major. Furthermore, in order to represent the state in the parallel key where equality holds as a set of scales, such as Clyd=ADor, Lyd in the mode system drawing section (M11-21), which is the second row from the top of the block, and Dor in the mode system drawing section (M11-31) on the parallel key side, which is the third row from the top, are placed so that they touch. The same method of representation is used for these 7 mode systems CM in the drawing sections of the first row (M11-10, M11-11), the third row (M11-30, M11-31), and the fourth row (M11-40, M11-41).

[0127] Next, we will explain an example of how to represent the H transformation that generates the multiplication 7CM. The H transformation that generates the multiplication 7CM is also represented by overlaying a square H transformation state drawing section (M11-22) of the key major key on top of the key major key mode system 7CM drawing section (M11-21), not completely covering it. This is the same for the first drawing section (M11-10 to M11-13), the third drawing section (M11-30 to M11-33), and the fourth drawing section (M11-40 to M11-43). As an example of the case where there is no mode system 7CM, Hm (=AeoHm) is the fourth from the left in Figure 6. An H box is displayed above Aeo. As an example of the case where there is a mode system 7CM, LydHM is the sixth from the left in Figure 6. An H box is displayed above Lyd, which is above Ion.

[0128] Next, we will explain an example of how to represent M-formation and W-formation to generate the multiplication system 7CM. M-formation and W-formation can be considered as difference changes from H-formation. For example, CMM={C, D, E, F, G, A♭, B♭} is obtained by flattening the B in CHM={C, D, E, F, G, A♭, B}. Therefore, in the case of M-formation and W-formation, not only the M / W-formation state drawing section (M11-23) but also the H-formation state drawing section (M11-22) are displayed adjacent to each other. This is also true for the first-row drawing section (M11-10 to M11-13), the third-row drawing section (M11-30 to M11-33), and the fourth-row drawing section (M11-40 to M11-43). An example of MM is the second from the right in Figure 6. There is also a box in the first row, but this will be explained later.

[0129] Among the 7CM multiplication systems, the following patterns are known to have a perfect match in terms of the central tone and constituent notes: MM=PhrMm, WM=PhrWm, LydMM=Mm, and LydWM=Wm. On the other hand, PhrMm and PhrWm have constituent notes that match those of MM and WM, which are tonalities centered on a fourth from the current central tone, while LydMM and LydWM have constituent notes that match those of Mm and Wm, which are tonalities centered on a fifth from the current central tone. For example, CWm={C, D, E♭, F♯, G, A♭, B}=CLydWM, which is a perfect match including the central tone, and GWM={G, A♭, B, C, D, E♭, F♯} differs in that its central tone is G (sol), but the set of constituent notes matches that of CWm and ClydWM. Visualizing the relationships between these closely related tonalities, such as CWm = CLydWM (≒GWM: the sets are the same, differing only in their tonality), is musically beneficial. To represent this, when WM, MM, LydMM, and LydWM are represented in the second line of the drawing section (M11-20 to M11-23), the equivalent 7CM are represented in the first line of the drawing section (M11-10 to M11-13), and the boxes are displayed so that they touch each other in the M or W drawing section (M11-23, M11-13). This visualizes what connects the 7CMs to each other. An example of MM and its equivalent PhrMm is the seventh from the left in Figure 6, and an example of LydWM and its equivalent Wm is the eighth from the left in Figure 6. The boxes representing M and W in the second row (M11-23) are placed adjacent to the first row, and the boxes representing M and W in the first row (M11-13) are placed adjacent to the second row, indicating that they are connected through M and W. The same applies to the third row of drawing sections (M11-30 to M11-33) and the fourth row of drawing sections (M11-40 to M11-43).

[0130] Figure 30 shows the confirmation of similar tonalities and its effect, including the visualization of modulation opportunities and actual chord conversion examples. Given a simple chord progression in C major (CNM) C→FM7→Bm7♭5→CM7→F(30-1), we can change it to similar 7CM chords such as CNM→CHM→CMM(=CPhrMm)→CPhrMm(=CMM)→CPhr(=FNm)(30-2). This makes it possible to modulate to FNm (F minor) via a complex chord progression such as C→FmM7→B♭7→C7→Fm(30-3). In this example, the similarity between 7CM chords such as CNM and CHM, CHM and CMM=CPhrMm (they are equivalent as sets in this case), and CPhrMm and CPhr≈FNm (they are equivalent as sets in this case) is utilized to achieve a seamless tonal change. The resulting CPhr Fm chord presents two distinct tonalities, CPhr and FNm, allowing for continuation as FNm (F minor), or as a borrowing from FNm to return to the key of C.

[0131] The visualization of tonality using the UI described above (M11) realizes the "means for visualizing the changing structure within a single key and the relationships between closely overlapping single keys, and for suggesting tonality, tonal changes, chord borrowing, or modulation opportunities" of claim 3.

[0132] As one embodiment of UI-based 7CM selection, we will describe the implementation of representation using a 7CM selection device (M12) as shown in Figure 7. The 7CM selection device (M12) is equipped with buttons that allow displaying or specifying the mode system 7CM and their H / M / W status. The luminosity values ​​for each 7CM selected here are calculated and managed in various information (241). Unless microtones are being dealt with, all 7CMs are basically defined as differences in semitone units (units where the luminosity value is 1), so each luminosity value will be either -1, 0, or 1. For example, in Mixo, the luminosity values ​​for the lower key (1st), upper key (5th), lower saturation (2nd), primary saturation (6th), and primary brightness (3rd) are all 0, while the primary luminance (7th) and upper luminance (4th) are -1. When Mixo is selected, processing is performed by the luminosity specification (G1) device using these values. Note that the above difference is the difference from Lyd, which is constructed by stacking perfect fifths from the 1st (the upper luminance, the 4th, is a semitone higher than NM), so the upper luminance value is set to -1.

[0133] The mode control 7CM and H / M / W button layout are displayed in four rows. The top two rows specify the key signature (including the parallel minor key) (M12-1), the bottom two rows specify the relative key (including the parallel major key of the relative key) (M12-2), and the middle two rows, spanning both rows, specify the key signature major / parallel minor key (M12-3). These sections are used to group and visualize the characteristics of the tonality. An example of a first-stage scale that is neither M12-1 nor M12-3 is CDor, where the tonic note is C and the key is minor. One example of a second-tier chord that is both M12-1 and M12-3 is the Clyd chord, which has C as its tonic note and is in a major key. An example of a third-tier key that is both M12-2 and M12-3 is AD or, where the central note is C and the relative key is temporarily aligned to create a sense of alignment. An example of a fourth-tier cadence that is neither M12-2 nor M12-3 is ANM, where the tonic is C and it can be temporarily perceived as being in the relative key. This is what is known in traditional music theory as a Picardy cadence.

[0134] Pressing the 7CM (mode) selection button (M12-511) with the mouse selects 7CM in the current block, and this is reflected in various glow-related information (241) in real time.

[0135] Pressing the 7CM (H / M / W system) selection button (M12-512) with the mouse applies or removes either H, M, or W to the 7CM in the current block, corresponding to the selected button, and this is reflected in various glow-related information (241) in real time. There are four rows of buttons; the first and third rows, representing minor keys, are labeled Hm / Mm / Wm respectively, and the second and fourth rows, representing major keys, are labeled HM / MM / WM. However, pressing any row will produce the same result.

[0136] Furthermore, as a control measure in Figure 7, among the three mode system 7CM selection buttons (M12-511) on the same stage, one or all of them are disabled to be enabled, and among the three H / M / W system selection buttons (M12-512), one or all of them are disabled to be enabled. For example, in the LydHM state where HM and Lyd are enabled in the second row, enabling MM will automatically uncheck the HM checkbox in the second row and enable MM. In other words, the second row will automatically be controlled from a state where only Lyd and HM are on to a state where only Lyd and MM are on. However, this control only applies within the same row; other rows are automatically controlled according to the 7CM at that time. In the example of the modified 7CM, LydMM, this is theoretically equivalent to Mm, so when Lyd and MM are on in the second row, the first row will automatically be configured to represent Mm, with only Aeo and Mm turned on.

[0137] Figure 7 shows a button that enables modulation to a similar key as described in claim 11. The key-parallel minor modulation button (M12-521) displays the relative minor as a label, and when pressed, modulates to the key that has the relative minor as its parallel minor. One example is the modulation from CNM to E♭NM (which suggests CNM in its relative key). The key-dominant modulation button (M12-522) displays the dominant key as a label and modulates to the dominant key when pressed. One example is the modulation from CNM to GNM. The key-subdominant modulation button (M12-523) displays the subdominant key as a label and modulates to the subdominant key when pressed. One example is the modulation from CNM to FNM. The parallel minor dominant modulation button (M12-524) displays the dominant of the parallel minor as a label, and when pressed, modulates to the key whose dominant is the parallel minor. One example is the modulation from CNM to GNM (suggesting ENm in the relative key). The parallel minor subdominant modulation button (M12-525) displays the subdominant key of the parallel minor as a label, and when pressed, modulates to the key that has the subdominant key of the parallel minor as its parallel minor. One example is the modulation from CNM to FNM (suggesting DNm in its relative key). The parallel major key modulation button (M12-526) displays the parallel major key as a label and modulates to the parallel major key when pressed. One example is the modulation from CNM to ANM. The 7CM adjacent key modulation button (M12-527) displays a label when a 7CM that matches the current 7CM in the scale set exists, enabling modulation to that key. For example, when Clyd is selected, GNM is suggested as a label, and when CMM (=CPhrMm) is selected, FNm is suggested, and pressing the button changes the tonic and relative key mode to achieve modulation.

[0138] As shown in Figure 4, the 7CM designated buttons (M12-511, M12-512) can be switched on / off using a shortcut from the PC keyboard (112).

[0139] This allows real-time on / off switching of the 7CM designated buttons (M12-511, M12-512) based on the input MIDI information (202).

[0140] Pressing the 7CM file loading button (M12-531) loads the 7CM file (1111), allowing you to specify the central tone, key / relative key status, and 7CM for each block number listed in the file. This information is saved in the various 7CM-related information (241).

[0141] Pressing the 7CM file write button (M12-533) will write the 7CM status information stored in various information (241) to an external 7CM file (1111).

[0142] When the current sequence block display / selection checkbox (S1-01) is enabled as shown in Figures 6, 8, and 9, or when the position is changed by operating each UI (S1-11 to S1-14) in Figure 13, the 7CM of the corresponding block is retrieved from various information (241) and reflected in the button state of the 7CM selection device (M12) in Figure 7.

[0143] The 7CM designation device (M12) with the UI described above realizes the following: the ability to arbitrarily designate a 7CM, which is a single tonality described in the 7CM theory, based on operation, input information, or the contents of an external file, as described in claim 4; the means to enable the management and reuse of tonality units and functional tone units based on the 7CM theory in a file format, as described in claim 10; the means to easily perform modulation by UI operation for tonalities calculated to be similar tonalities in the 7CM theory, as described in claim 11; and part of the ability to set and manage tonality information, functional tone information, 7CM information, luminosity value information, performance mode, and sequence designation information according to the input MIDI note information.

[0144] Figure 8 illustrates one embodiment of the processing of the functional sound specification device (F1) from Figures 2 to 4 using the UI, specifically the visualization and setting of functional sound progression by the functional sound progression specification unit (F11). The functional sound progression specification unit (F11) represents the defined functional sound progression as a two-dimensional graph (F11-2) on layers representing functions (F11-11 to F11-14). This graph is then stored as functional sound progression in various information (241) by pressing the functional sound progression reflection button (F11-3) after mouse operation.

[0145] The functional sounds defined for the display numbers on the sequence block number display label (S1-02) are displayed as a two-dimensional line graph for functional sound definition (F11-2) at the locations of the numbers indicated on the functional sound progression visualization / drawing unit (F11-1).

[0146] The line graph for defining functional sounds (F11-2) can be drawn with the mouse, and pressing the functional sound progression reflection button (F11-3) saves it to various information (241).

[0147] The functional tone layers (F11-11, F11-12, F11-13, F11-14) are displayed as layers, with functional tones possessing the D, L, S, and T functions arranged in groups. By arranging them from top to bottom in the order of D, L, S, and T, the sense of musical resolution or progression when moving to T is visualized as an emotional line graph, likened to the emotional fluctuations of landing on the ground at T from a high point. For example, in C major (CNM), the progression Am→C (function tone 6→function tone 1) is a transition from the S layer to the T layer, and is depicted on the graph as a gradual progression. The progression from Dm to C (function tone 2→function tone 1) in the L layer is a stronger progression, as can be seen from the difference in the graph, and this is actually considered a strong progression in conventional music theory as a stepwise progression. Furthermore, the progression from G to C (function tone 5→function tone 1) in the D layer is an even stronger progression, as can be seen from the larger difference in the graph, and this is considered a strong progression in conventional music theory as having an even stronger sense of resolution and progression. By plotting the graph on the functional layers, the strength of the sense of progression is visualized through the difference in the graph.

[0148] The functional tone progression specification section (F11) has three or more types. One displays functional tone progressions within the key, one displays functional tone progressions within the relative key, and the other displays functional tone progressions within a single key centered on another arbitrary degree, with the key displayed on the key signature label (F11-5) for each. The central tone of an arbitrary key can be changed using the central tone specification knob (F11-4) for the displayed key, and the 2D line graph for functional tone definition (F11-2) is redrawn in real time by dragging the knob with the mouse. Figure 27 shows an example of a chord progression, illustrating that even with the same chord progression, the graphs differ completely depending on which note (1st degree) is chosen: a graph centered on the tonality C (E1-411), a graph centered on the relative key A (E1-421), and a graph centered on F with an arbitrarily specified 4th (E1-431). By simultaneously visualizing a single tonality from the perspective of multiple tonalities, it becomes possible to examine the borrowing state, as described in conventional music theory, by peeking into the functional tone progressions of other tonalities.

[0149] For example, the chord progression Am→G7→C→F in C major (CNM) is represented as 6→5→1→4 in terms of the tonal functional tones. In the relative key, the chord progression itself is Am→G7→C→F, but because the tonal A of the relative key Am is considered as the 1st degree, the functional tone progression becomes 1→7→3→6, and the appearance of the functional tone definition line graph (F11-2) is different. If the knob for specifying the central tone of the displayed tonality (F11-4) is set to 4, it becomes like looking into a single tonality where the 4th degree F is the central tone, so the chord progression Am→G7→C→F becomes a functional tone progression of 3→2→5→1, and the appearance of the functional tone definition line graph (F11-2) is yet again different. On a separate single tonality, the key of F, the chord progression that is frequently used and called the two-five-one (2→5→1) in conventional music theory is visualized, and opportunities for borrowing from the key of F are made visible. Here, as an example of an approach to arranging a chord progression into something complex like Am→Gm7♭5→C7♭9→F, the conventional approach required (1) to make C7 the secondary dominant, changing the preceding chord to Gm7 to form a two-five-one progression and maintaining tonal consistency (B♭), (2) to consider the tensions that can be added to C7, and (3) to adopt the ♭9 that has been felt to sound good in the past as a tension for C7. This required musical knowledge (1) and (2), as well as the intuition and experience (3) for that chord. However, with the present invention, these are no longer necessary. First, by examining the two-dimensional broken line graph (F11-2) in the key of F using the functional tone progression specification section (F11) in Figure 8, it can be confirmed that "when viewed in the key of F, the functional tone progression is a progression that lands on T as 2→5→1." Looking at Figure 7, the M12-523 on the right side is displayed as F, and by selecting Mixo on the M12-511 button located opposite it, G7 is converted to Gm7, and C is converted to C7 when a 7th is added. Furthermore, in order to "express a subtly melancholic atmosphere, so I'll change Mixo to H," by selecting HM on the M12-512 button in the row of the selected Mixo in Figure 7, 7CM becomes CMixoHM, and as a result, Gm7 is converted to Gm7♭5, and C7 is converted to the complex chord C7♭9 when a 9th is added. Even such complex chords can be actively and intentionally adopted visually and intuitively by the user, without relying on knowledge or chance.

[0150] Furthermore, when not in a parallel key state, the priority tonality state visualization drawing frame (F11-6) of the key's priority tonality state is highlighted in the functional tone progression specification section (F11) UI (F11) of Figure 8, which has three or more elements: key, parallel key, and arbitrarily specified key. When in a parallel key state, the same drawing frame (F11-6) of the parallel key is highlighted to make the priority single tonality to be viewed stand out.

[0151] As shown in Figure 4, a shortcut from the PC keyboard (112) applies and updates the specified functional sound to the currently selected sequence block. If there are any active checkboxes (S1-01) in the current sequence block display / selection checkboxes (S1-01) shown in Figure 8 on the display screen, the functional sound of the corresponding block is updated and reflected in the functional sound definition line graph (F11-2) in real time.

[0152] As shown in Figure 10, pressing the function sound progression file loading button (F12-31) loads the function sound progression file (1112) as shown in Figure 3, stores the function sounds for each block number in the various information (241), and redraws the line graph for function sound definition (F11-2) in Figure 8 and the function sound progression connection visualization UI (F12) in Figure 9 in real time.

[0153] As shown in Figure 10, the functional sound progression is written to file (1112) by pressing the functional sound progression file loading button (F12-33).

[0154] As described above, the processing of the functional sound progression specification unit (F11) that visualizes and sets functional sounds realizes the means described in claim 5, which "sequentially manages a sequence of functional sounds according to the operation, input information, or contents of an external file, and includes a display unit that visualizes the in-tonality function of the functional sound in a two-dimensional graph, and displays the progression status, chord borrowing, or modulation opportunities in multiple tonalities in real time," and the means described in claim 10, which "enables the management and reuse of tonal units and functional sound units based on the 7CM theory in file format."

[0155] Figure 9 illustrates the visualization of inter-functional sound attraction using the functional sound progression connection visualization UI (F12) as one embodiment of the visualization of attraction between functional sounds to assist in the operation of the functional sound progression specification section (F11) in Figure 8. In the attraction theory of 7CM theory, not only the attraction state with the next adjacent functional sound but also the reserved attraction, which is the attraction with the functional sound two steps ahead, is considered important. For example, in a progression such as G→Em7→Am, Em7→Am is a D attraction connection, but the L attraction from the preceding G to the final Am is also considered important. To express these concepts, the function connection status display areas (F12-1, F12-2), which are box-shaped objects representing each function sound, are arranged in a zigzag pattern from left to right in a time series. The relationship with the function sound one step ahead vertically and the relationship with the function sound two steps ahead horizontally are expressed by the presence or absence of display on each function connection status (F12-11 and F12-22, F12-12 and F12-21 touching vertically, and F12-14 and F12-13, F12-24 and F12-23 touching horizontally), which are boxes representing the connection surfaces of the function connection status display areas (F12-1, F12-2), and the type of attraction (D attraction, L attraction) is expressed by the color of the boxes.

[0156] The connections from the function itself (F12-12 if within the first function connection status display area (F12-1), F12-22 if within the second function connection status display area (F12-2)) and the connections to the function itself (F12-11 if within the first function connection status display area (F12-1), F12-21 if within the second function connection status display area (F12-2)), which are drawn touching above and below, indicate the attractive force between the function sounds before and after the connection. If a D attractive force or an L attractive force exists, it is colored in a different, visible color such as blue or yellow, respectively. If neither exists, nothing is displayed.

[0157] The same applies to the connections between functional sounds that skip one note, which are the hold attraction in the 7CM theory. The hold connections from the self-function (F12-14 if in the first-stage function connection status display area (F12-1), and F12-24 if in the second-stage function connection status display area (F12-2)) and the hold connections to the self-function (F12-13 if in the first-stage function connection status display area (F12-1), and F12-23 if in the second-stage function connection status display area (F12-2)), which are drawn adjacent to each other on the left and right, indicate the attraction between the functional sounds before and after the hold connection. If a D attraction or L attraction exists, it is colored in a different, visible color such as blue or yellow, respectively, and nothing is displayed if it does not exist. This visualizes the connection state of functional tones and the psychological expectations associated with the set chord progression. For example, in C major, the chord progressions F→Em and F→Am both have functional tones of 4 (S layer)→3 (S layer) and 4 (S layer)→6 (S layer), respectively. In the functional tone progression specification section (F11) of Figure 8, both are represented as chord transitions on the same S layer. However, in the functional tone progression connection visualization section (F12) of Figure 9, the former is shown to have an L-attraction force, while the latter does not. Therefore, it becomes possible to select and design chords with a stronger sense of connection and progression.

[0158] The function connection status display areas (F12-1, F12-2), which are boxes representing each function sound, are colored with a color representing the function of each function sound, and the function name and function sound display labels (F12-10, F12-20) display the function name and function sound in the key. Unlike the function sound progression specification section (F11), which displayed functions in different keys using multiple UIs, the function sound progression connection visualization section (F12) does not depend on the key but on the relative relationship between function sounds, so it has only one UI, and the function name and function sound display labels (F12-10, F12-20) display only the information in the key as a representative example.

[0159] As described above, the functional sound progression connection visualization unit (F12), which visualizes the attractive force between functional sounds one and two steps ahead, is a means of claim 6 that "visualizes the attractive force and sustained attractive force between functional sounds based on the connection state between one or two steps ahead of sequentially managed functional sounds, and visualizes the sense of progression expectation that functional sounds in a chord progression have."

[0160] As shown in the example in Figure 11, various information such as the central tone number (242-12), key name (241-11), 7CM (241-2), relative key status (241-3), and performance mode (P1-1) are displayed.

[0161] The display of various information (241) using the UI described above constitutes a means for "managing the characteristics of music (at least one of the following: brilliance, 7CM, structure of 7CM, relationships between 7CM, function of functional sounds, attraction between functional sounds, and sense of anticipation of chord progressions)" as described in claim 1.

[0162] Another example of the present invention, a second embodiment that realizes claim 12, is shown. The first configuration applies light management and tonal control based on the 7CM theory to a game sound engine (W1) to realize the music control system described in claims 1, 2, and 12, as shown in Figure 21. This is an example where light is interactively controlled on the sound engine side rather than the game engine side. Here, Wwise (W1) is used as an example, but it is not limited to this sound engine, as it has the ability to receive status and game parameters from the game engine and to change the pitch of the output sound according to the parameters. Furthermore, this method can also be implemented on the game engine side.

[0163] In this embodiment, the Wwise or other game sound engine (W1) acquires the luminance state (W1-272) sent from the game engine (W2) and changes the luminance value in the playback music according to the state value (W1-2721, W1-2722). For example, if the luminance state (W1-272) is Low (W1-2722), the main luminance, which is the luminance of the 7th, is treated as -1.0, and the pitch of the Actor-Mixer (W1-27) for 7th sound generation is lowered by a semitone. As a result, when MIDI note data (W1-1117, W1-1127, W1-1137) consisting only of the 7th for instruments P01, P02, and P03, which are sounded through the Actor-Mixer (W1-27) for 7th sound generation, are sounded, if sound is currently being generated, the notes will continuously change to diminished 7th notes, and all subsequent sounds will be sounded as diminished 7th notes. This method enables interactive changes in the musical atmosphere that correspond to the game's situation.

[0164] Music Playlist (W1-1) and Music Segment (W1-11) manage MIDI note data divided into degrees from 1st to 7th for each instrument P01, P02, and P03 (W1-1111 to W1-1117, W1-1121 to W1-1127, W1-1131 to W1-1137, etc.), and apply tonal changes based on 7CM theory. The MIDI targets for these sounds are set in BlendContainers that manage the audio files for each instrument, which are placed within Actor-Mixers Deg1 to Deg7, prepared according to the degree. By controlling the pitch in units of ActorMixers from Deg1 to Deg7, the tonality of all instruments can be synchronized. This embodiment utilizes the same technical foundation (7CM theory, management of glow parameters) as the VSTi implementation (first embodiment) while realizing interactive tonal changes specific to the game environment.

[0165] The second embodiment of the second set of features describes an embodiment that realizes an experience that links the perceived light in front of the user's eyes with the perceived light of the music they are listening to, based on color space information detected by a webcam. There is a Pd-extended+GEM environment as an extension of Puredata, a music environment that runs on a PC. As described in Non-Patent Literature 3, color information can be acquired using webcam input as a source, and for example, luminance can be calculated from the YUV color space of that source, lightness from the converted RGB color space, and saturation from the converted HSV color space. This is just one example, but applications are not limited to this, such as simply using the individual values ​​of RGB. Instead of applying these input values ​​to the entire song or to the scales and timbres of specific instruments, they are applied to each degree within the tonality of the musical tones based on the 7CM theory for tonality changes. For example, luminance information, brightness information, and saturation information are plotted on the primary luminance (7th degree), primary brightness (3rd degree), and primary saturation (6th degree) of the 7CM theory, respectively. When playing music from Puredata, the luminosity values ​​are converted according to each degree, allowing for interactive musical expression based on the luminosity of the video information (in this case, webcam information) that the user is experiencing.

[0166] The third embodiment of the second model describes a form based on smartphone tilt information. Non-patent document 4 describes a site with a MobMuPlat application that runs on smartphones. Puredata can operate on smartphones via this application, and MobMuPlat has a process for acquiring the smartphone's tilt value. This information is passed to Puredata, which then modifies the tonal structure of the music being played. By mapping the vertical direction of tilt to the primary brightness value and the horizontal direction to the primary saturation value, the tonality of the music is interactively changed according to the smartphone's tilt.

[0167] The interactive playback of luminosity and 7CM in response to user behavior and external information, as described in the first to third embodiments of the second embodiment above, realizes the "management of music characteristics" of claim 1, the "dynamic retention and setting of luminosity information in response to operation, input information, or the contents of an external file" of claim 2, and the "music control method characterized by performing complex melody conversion or chord conversion based on 7CM or luminosity values ​​selected by the user or interactively calculated based on inputs such as user behavior or environmental changes" of claim 12.

[0168] Another example of the present invention, a third embodiment, is shown below. The first configuration applies chord progression control based on 7CM theory to the game engine (O1-1) and the game sound engine (O1-2) to realize the music control system described in claims 1, 4, and 13, as shown in Figure 22. While an example of implementation on the game engine side is given, implementation on the game engine side is not mandatory. Any control environment capable of generating MIDI information using the game state as input can be used, for example, by processing on the game sound engine side (such as a custom script within Wwise), and the configuration is not limited.

[0169] In this embodiment, the game engine (O1-1) performs playback processing (O1-111) of a MIDI sequence (O1-112) to be played repeatedly, and the game audio engine (O1-2) plays it back. During this process, it is dynamically determined (O1-12) whether or not to generate connections defined by 7CM theory in the chord progression according to the game situation, and the next chord to be played is interactively determined (O1-13 to O1-16), and then registered in the MIDI sequence information (O1-112) (O1-17). When determining the chord, since some 7CM chords may cause discomfort to the user, appropriate 7CM chords are calculated (for example, if the 1st and 5th degrees of a chord are a diminished fifth dissonance, a 7CM chord is selected and calculated that changes one of them to a perfect fifth), and the system is managed to avoid giving the user unnecessary irritation. This allows the user to experience background music that is integrated with the game's progress.

[0170] The second part of the third embodiment describes a form based on user movements detected by a webcam. Similar to the second part of the second embodiment, the Pd-extended+GEM environment is used. As described in Non-Patent Literature 3, motion detection is possible using webcam input as the information source. By using the pix_movement object, which calculates the frame difference of the video, and the pix_blob, which finds the centroid of the difference image, it is possible to determine where on the screen the movement occurred. Based on this movement position information, movement to the right of the screen is treated as a connection instruction, and movement to the left is treated as a stagnation instruction. Similar to the first part of this third embodiment, the playback processing of a MIDI sequence that should be played repeatedly is performed on Puredata, and the next sequence is always calculated / defined. When the user's connection / stagnation instruction value is obtained, the mode (connection / stagnation) of the next sequence to be created is switched. This makes it possible for the user to give instructions such as moving on the right side of the screen when they want to hear a constantly progressing chord progression, or moving on the left side when they want to stagnate, and to control the connection between chords accordingly.

[0171] The third embodiment of the third configuration describes a configuration based on smartphone shake detection. Similar to the third configuration of the second configuration, MobMuPlat is used, specifically its shake detection function. This shake information is sent to Puredata, which counts it. The speed of the shake is evaluated by comparing it with the count of the metro object. If it is faster than a certain value, it is treated as a connection instruction; if it is less, it is treated as a stagnation instruction. Similar to the first configuration of this third configuration, the playback process of a MIDI sequence to be played repeatedly is performed on Puredata, and the next sequence is constantly calculated / defined. When the user's connection / stagnation instruction value is obtained, the mode (connection / stagnation) of the next sequence to be created is switched. This enables interactive control of connections between codes according to whether the user's walking speed is fast or slow.

[0172] The process of interactively determining which chord progressions to connect and which not to connect in response to user behavior and external information, as described in the first to third embodiments of the third embodiment above, selecting a suitable 7CM according to the chord characteristics, and applying it to the note numbers that make up the chord, realizes the "management of musical characteristics" of claim 1, the "arbitrary specification of a 7CM which is a single tonality described in 7CM theory based on operation, input information, or the contents of an external file read" of claim 4, and the "music control method characterized by performing complex chord progression arrangements using functional sounds selected by the user or interactively calculated based on inputs such as user behavior and environmental changes" of claim 13. [Examples]

[0173] 1. An example of the first embodiment is shown.

[0174] As an example of an implementation method, the processing described in claims 1 to 11 is implemented using PureData (Pd). Pd can be called as a VSTi from various DAWs (for example, Cubase 13) and other VST-compatible host applications via Plugdata. This VSTi can input and output various MIDI information, and the output can be received as input to a MIDI track on the DAW side. As implementation examples, the website linked to the URL in Non-Patent Document 5 contains "(1) 7CM Change Sample" which shows the visualization of functional sounds and 7CM settings, and "(2) Glow Sensitivity Change Sample" which shows a microtone change sample where the glow sensitivity gradually changes.

[0175] Figure 23 shows an example of the implementation method using a two-instrument configuration of bass and harmony. Two VSTi (this music processing system) for performance are placed in the VSTiRack, and from now on we will refer to them as VSTiB (E1-131) and VSTiH (E1-231), respectively. Prepare a MIDI track for sequencing and a MIDI track for playback MIDI notes for both the bass and harmony. Name the bass tracks BSeq (E1-111) and BNote (E1-112), and set their output destination to VSTiB (E1-131). Name the harmony tracks HSeq (E1-211) and HNote (E1-212), and set their output destination to VSTiH (E1-231). Furthermore, for the VSTiB's MIDI output channels Ch1 to Ch7 and Ch8, eight MIDI tracks are prepared with the names BCh1 to BCh7 (E1-141 to E1-147) and BCh8 (E1-148) as inputs. Similarly, eight tracks are prepared for the VSTiH's eight output channels with the names HCh1 to HCh7 (E1-241 to E1-247) and HCh8 (E1-248). This separates the VSTi output results into eight tracks: one for each degree from 1 to 7 and another for all degrees. We also provide VSTis to play the MIDI information output by BCh1 to 8 and HCh1 to 8, but you can also assign them to each channel (E1-151 to E1-157, E1-158) within a multitimbral sound source (E1-15), as in the bass example, or you can prepare a separate VSTi for each channel (E1-251 to E1-257, E1-258), as in the harmony example.

[0176] Figure 24 shows an example of pre-playback preparation in the key editor image for each track. A note (E1-331) with note number 125 and velocity 1 is placed in the first measure (E1-32) of BSeq (E1-111). When the note number is 125, the velocity value is used to specify the block's sequence number, so when the VSTiB receives this velocity value of 1, the block's sequence number is set to 1. Also, a note (E1-332) is placed at note number 96, specifying that the central tone is C.

[0177] As an example of preparation for changing chords every two beats starting from the second measure, as shown in Figure 24, place note number 127 and notes with a velocity of 101 or higher (E1-35) on the BSeq at two-beat intervals thereafter, on the third beat of the second measure, the first and third beats of the third measure. Since this note is treated as control information to advance to the next sequence within the music control system, each time the DAW plays this MIDI note, the block in the music control system will advance by one. Since the sequence number of the block is specified as 1 at the start of the first measure, no specification is needed at the beginning of the second measure. Copy the BSeq track created up to this point to the HSeq track (E1-211 in Figure 24). Furthermore, within the first measure, place note number 122 (E1-333), which indicates the performance mode of bass mode, on the BSeq, and place note number 120 (E1-334), which indicates the performance mode of Harmony mode, on the HSeq to specify the respective performance modes.

[0178] As an example, we will show the process of inputting a Canon chord progression in C major and arranging the chord progression. There are various versions of the Canon chord progression, but here we will use the progression (1)CM7→(2)CM7→(3)G7→(4)G7→(5)Am7→(6)Am7→(7)Em7→(8)Em7→(9)FM7... in 2-beat units as the initial state before arrangement.

[0179] As shown in Figure 25, first load the original functional sound progression file (E1-411). The file format consists of block sequence numbers and functional sounds separated by half-width spaces, with semicolons at the end of each line, allowing it to be loaded line by line in Pd. In this example, it shows the functional sound progression (1)1→(2)1→(3)5→(4)5→(5)6→(6)6→(7)3→(8)3→(9)4.

[0180] Next, load the original 7CM file (E1-412) as shown in Figure 25. The file format consists of the block sequence number, tonic note number, relative key status, mode system 7CM, and H / M / W mode, separated by half-width spaces and ending with a semicolon, allowing it to be loaded line by line in Pd. In this example, sequences 1 to 4 and 7 to 9 represent a 7CM called CNM, which has a tonic note of 96(C), is Ion on the tonic side, and is not H / M / W. Sequences 5 and 6 represent a 7CM called ANm, which has a tonic note of 96(C), is Aeo on the relative key side, and is not H / M / W.

[0181] As shown in Figure 24, MIDI information for sound generation is entered into BNote (E1-112) and HNote (E1-212). In BNote, the note C (note number 36) is repeatedly entered as a half note every two beats (E1-36). In HNote, the notes C (note number 48), D (note number 50), E (note number 52), and F (note number 53) are repeatedly entered as half notes every two beats (E1-37). In the track created here, when a normal MIDI instrument is played, only the note C is always played in BNote and only the C-D-E-F chord is always played in HNote. When this is played back through VSTiB (E1-131) and VSTiH (E1-231) as shown in Figure 23, the conversion shown in Figure 25 occurs. The output result (E1-1321) of the conversion process (E1-131) for NoteBN (E1-122) in Bass mode is (1) Do → (2) Do → (3) Sol → (4) Sol → (5) La → (6) La → (7) Mi → (8) Mi → (9) Fa, and the output result (E1-2321) of the conversion process (E1-231) for NoteHN (E1-222) in Harmony mode is (1) CM7 → (2) CM7 → (3) G7 → (4) G7 → (5) Am7 → (6) Am7 → (7) Em7 → (8) Em7 → (9) FM7. At this point, the bass and chord progressions before arrangement have been constructed using simple MIDI note information.

[0182] Figure 23 shows the flow of sound production in Bass mode. The notes converted by the process shown in Figure 25 are output from channels corresponding to their respective degrees, and all notes are output together. For example, from BCh1 (E1-141), the notes from (1) C to (2) C, which are the 1st degree, are output, but the notes from (3) G to (9) F are not sounded because they do not represent the 1st degree. For BCh7 (E1-147), there are no notes corresponding to the 7th degree between (1) C and (9) F, so they are never sounded. For BCh8 (E1-148), since it is a channel that outputs all degrees, all of the notes from (1) C → (2) C → (3) G → (4) G → (5) A → (6) A → (7) E → (8) E → (9) F are output.

[0183] Figure 23 shows the flow of sound when Harmony mode is played. The notes converted by the process shown in Figure 25 are output from the channel corresponding to the degree, and all notes are output together. The converted chords (E1-232) are CM7 (C E G B), G7 (G B D F), Am7 (A C E G), Em7 (E G B D), and FM7 (F A C E). The root note C is included in 5 places: (1) and (2) CM7, (5) and (6) Am7, and (9) FM7. The 7th note B is included in 6 places: (1) and (2) CM7, (3) and (4) G7, and (7) and (8) Em7. Therefore, for example, HCh1 (E1-241) outputs a C note at timings (1), (2), (5), (6), and (9), and HCh7 (E1-247) outputs a B note at timings (1), (2), (3), (4), (7), and (8). HCh8 (E1-248), which is a channel that outputs all degrees, always outputs all chord tones.

[0184] So far, we have shown examples of the implementation of claims 1, 4, 5, 8, 9, and 10.

[0185] Furthermore, the functional sound progression is modified using claims 5 and 6. Blocks (2) to (9) of the functional sound progression up to this point are represented by a two-dimensional line graph and an attraction visualization section as shown in Figure 27. Since block (1) has the same content as (2), the eight display states from (2) to (9) will be explained.

[0186] The functional characteristics shown in Figure 27 include a sharp rise in the basic tone 2D line graph (E1-411) from block (2) to (3), a transition from the L layer to the T layer from block (4) to (5) in the parallel tone 2D line graph (E1-421), and a transition from the L layer to the T layer from block (8) to (9) in the arbitrary tone 2D line graph (E1-431) displayed using the F key. This time, we will make changes to make this transition smoother and more dynamic.

[0187] The attractive force perspective in Figure 27 shows that, as shown in E1-441, there is a retained L-force from block (3) to block (5) and a direct L-force from block (4). Also, as shown in E1-442, there is a retained L-force from block (4) to block (6), and as shown in E1-443, there is a retained L-force from block (7) to block (9) and a direct L-force from block (8). This modification will be made so that these connections are not severed.

[0188] As shown in Figure 28, the functional sound of block (3) is changed to 7, the functional sound of block (4) to 3, the functional sound of block (7) to 5, and the functional sound of block (8) to 1. The connections of each attraction force (E1-541, E1-542, E1-543) in the attraction visualization section are maintained as in the original state, and in particular, from block (4) to (6), (E1-542) has changed from an L attraction force to a D attraction force, indicating a stronger connection. Furthermore, new connections have been added in places such as from block (2) to (3) (E1-544), from (3) to (4) (E1-545), from (5) to (7) (E1-546), from (6) to (7) (E1-547), and from (7) to (8) (E1-548), confirming that the sense of progression has been made stronger.

[0189] Furthermore, in the two-dimensional line graph of the key (E1-511), the strongest sense of resolution is formed from block (7) to (8) with a landing from the D layer to the T layer, in the two-dimensional line graph of the parallel key (E1-521), the strongest sense of resolution is formed from block (4) to (5) with a landing from the D layer to the T layer, and in the two-dimensional line graph of the F key with the arbitrary key set a fourth apart (E1-531), the strongest sense of resolution is formed from block (8) to (9) with a landing from the D layer to the T layer. Looking at it in more detail, in the parallel key and the F key, looking back from the previous step, both form a functional tone progression of 2→5→1, which is called a two-five-one in conventional music theory and is a frequently used progression. This is a borrowed two-five-one from another key.

[0190] Up to this point, the chord progression is (1)CM7→(2)CM7→(3)Bm7♭5→(4)Em7→(5)Am7→(6)Am7→(7)G7→(8)CM7→(9)FM7. Examples of the implementation of claims 1, 5, and 6 are shown.

[0191] Next, Figure 29 shows the process of changing to 7CM. Using the 7CM setting device (M12) in Figure 7, the settings are changed to (3)CMM, (4)AHm, (7)CMixo, and (8)CMixoHM. Figure 29 shows the 7CM visualization section (E1-62) which visualizes the 7CM state for blocks (2) to (9). Block (3) uses MM with reduced main brightness (7 degrees) and main saturation (6 degrees) to create a sense of vastness, and in block (4), a strong progression from the D layer in parallel tone was observed when the functional sound was changed (from block (4) to (5) in E1-521 of Figure 28), so Hm on the parallel tone side was used to bring out the color of the parallel tone layer strongly, thereby creating a sharp, refined, and tight darkness. Similarly, as mentioned above, a two-five-one progression on the F key side can be confirmed from block (7) to (9) (from block (7) to (9) of E1-531 in Figure 28), so a Mixo mode that leans towards that tonality is selected, and in order to create a sense of melancholy within that tonality, block (8) in Figure 29 is further elevated to MixoHM. By performing the 7CM settings while referring to the respective two-dimensional line graphs (E1-511, E1-521, E1-531 in Figure 28) manipulated when changing the functional sound, borrowing from other keys becomes easier, allowing for more effective and efficient chord progression analysis.

[0192] Figure 26 shows the output of the functional tone progression file and the 7CM information file at this point. The functional tone progression file will be the content of E1-711, and the 7CM information file will be the content of E1-712. NoteBN (E1-122) is converted to (1) Do → (2) Do → (3) Si♭ → (4) Mi → (5) La → (6) La → (7) Sol → (8) Do → (9) Fa (E1-1322), and NoteHN (E1-222) is converted to (1) CM7 → (2) CM7 → (3) B♭7 → (4) E7 → (5) Am7 → (6) Am7 → (7) Gm7 → (8) C7 → (9) FM7 (E1-2322). However, in the full degree output from BCh8 (E1-148) and HCh8 (E1-248) in Figure 23, these are output as note numbers. However, for BCh1 to BCh7 (E1-141 to E1-147) and HCh1 to HCh7 (E1-241 to E1-247), which are output in degree units, the 7CM state is reflected by outputting pitch bend information that increases or decreases in semitone increments when the 7CM information is changed, and the note information output is the information after the functional sound has been reflected.

[0193] Furthermore, in this example, the code (C7 of E1-2322) output by block (8) of NoteHN (E1-222) in Figure 26 will be the same C7 code in both Mixo and MixoHM. However, if this 7CM output file is reused to add a Melody mode to another track, for example, the moment the melody touches the 9th degree (D) of the chord, the latter 7CM MixoHM conversion will cause it to be pronounced as a D♭. By enabling reuse in units of 7CM information, it becomes possible to create a sense of tonality for a moment by another part without constantly emphasizing the tonality of the chord. In other words, while conventional technology required conscious design of the consistency between chord tones and melody tones, by separating and reusing / designing in units of tonality, it becomes possible to create music without being conscious of the delicate tonality between these instrument parts.

[0194] Up to this point, we have shown additional examples of the implementation of claims 1, 4, and 10.

[0195] Furthermore, the luminosity change device (G1-1) in Figure 5 is used to make subtle luminosity changes. Here, the main luminosity setting slider (G1-107) on the VSTiH for harmony is set to 0.1. As a result of this change, a bend value of 0.1 semitones is sent as pitch bend information to HCh7 in Figure 23, as shown in 253 of Figure 2, and from then on, all notes produced from this channel will be produced 0.1 semitones higher unless a new setting such as 7CM is specified. This results in the output of chords with slightly higher luminosity, giving the overall tone a somewhat glittering feel. Note that this does not apply to the bass output unless the VSTiB for bass is operated, nor does it apply to HCh8, which outputs all degrees for harmony. However, if a value such as 0.6 semitones is set, even for output channels for all degrees such as HCh8, the 7th note will be output with a note number that is a semitone higher due to rounding.

[0196] Up to this point, we have provided additional examples of the implementation of claims 1, 2, and 7.

[0197] 2. An example of use in the second embodiment is shown.

[0198] The second embodiment (claim 12) shows an example of the form described in the first part. Audiokinetic's Wwise Samples, available as of April 2025 at the URL in Non-Reference 6, include a game sample called Cube, and in its demo's map called Nudist, there is a place where the user can freely dive, like a swimming pool. The Wwise audio engine is designed so that when the player enters this water, a value of Yes is sent to the PlayerInWater State, and when the player exits the water, a value of No is sent.

[0199] In the first example of the second embodiment described in Figure 21, the State named "luminance" is renamed to "PlayerInWater," and the value of "High" is changed to "No," and the value of "Low" is changed to "Yes." This enables interactive changes in the musical tone, where the main luminance of the BGM decreases when the player enters water during the game, and returns to its original level when the player exits the water. This BGM, linked to the visual information obtained from the game, enhances immersion in the game. As an example of implementation, Non-Patent Document 5, under "(3) Interactive Change Sample in Games," contains an interactive BGM tone change sample using Wwise and a game demo. In this sample, as an example of manipulating multiple luminosity values ​​simultaneously, the primary luminosity (primary brightness of 3 degrees, primary saturation of 6 degrees, and primary brightness of 7 degrees) is increased or decreased depending on whether the user is underwater or outside of water.

[0200] In the second embodiment (claim 12), the second example uses the embodiment described in the second of the second embodiment, and when a user wearing a webcam moves to a bright environment, the main brilliance of the music (main brightness of 3 degrees, main saturation of 6 degrees, main luminance of 7 degrees) is subtly increased to create a dazzling effect.

[0201] As the third embodiment of the second embodiment (claim 12), the embodiment described in the third of the second embodiment is used, and the vertical tilt as described is mapped to the main brightness (3 degrees) and the horizontal tilt to the main saturation (6 degrees), and these values ​​are reflected in the pitch of each channel of Pd to control the tonality. This allows the user to easily change the atmosphere of the music by manipulating the angle of their smartphone according to their mood, enabling personalized musical expression.

[0202] 3. An example of use in the third embodiment (claim 13) is shown.

[0203] The third embodiment (claim 13) uses the embodiment described in the first of the third embodiments. When calculating future performance information, the game situation is always referenced (O1-12) and the calculation is linked to the presence or absence of the gravitational pull of the 7CM theory. When generating chords, 7CM selection and chord calculation are performed by excluding 7CMs that create dissonance within the chord, such as diminished fifths / augmented fourths or minor seconds, or by excluding the notes themselves, thereby controlling the generation of stimuli that are not directly related to the game.

[0204] The game itself could be something like solving puzzles and gradually escaping. While the game is stalled, the connection check (O1-12) will always show No, and a function tone that does not connect to the function tone of the currently playing chord will be selected (O1-13). As an example, a function tone a third above, a third below, or a fourth below the function tone of the current chord will be calculated.

[0205] Next, to calculate the chord, a triad is constructed by stacking thirds, centering on the next functional tone in the current 7CM. If it includes dissonances such as diminished fifths, note conversion candidates are calculated using other 7CM candidates. From several patterns that resolve the dissonance, a 7CM that is modal or a 7CM with the mode H / M / W is selected (O1-15), and the chord is calculated (O1-16). The 7CM selected here will be used as the 7CM for subsequent calculations. For example, if a functional tone seventh is determined in CNM, the chord is Bdim (B-D-F), and B and F are dissonant with a diminished fifth. However, there are various 7CMs that keep F the same and flatten B (CMM, CMixo, etc.) or 7CMs that keep B the same and sharpen F (CLyd, ADor, etc.), and the choice is made from these. The chord obtained here is registered in the sequence to be played next (O1-17), and will be played from the repeat sequence playback process (O1-111).

[0206] As this process is repeated, when one puzzle is solved, the connection check (O1-12) indicates "Yes," and a functional tone to connect to the functional tone of the currently playing chord is selected (O1-14). Specifically, a second above, a second below, or a fifth below the functional tone of the current chord is calculated. Subsequent processing is performed in the same way as the stalled 7CM selection (O1-15) and chord calculation (O1-16), and is reflected in the performance.

[0207] By performing the above processes, users will feel as if the music itself is progressing in accordance with the situation as they solve a puzzle and advance through the game, creating a sense of immersion in the game's progression.

[0208] The third embodiment (claim 13) uses the second embodiment described in the second part of the third embodiment. By performing motion detection definitions and corresponding mappings as described in the embodiment, it is possible to create a sense of progression that corresponds to the movements of the user to whom the webcam is pointed. For example, in a video of eating, the video can progress when the chopsticks in the right hand move frequently, and pause when the rice bowl in the left hand is placed on the table. This can be used in a variety of scenes.

[0209] The third embodiment (claim 13) uses the embodiment described in the third part of the third embodiment. By performing the mapping as described in the embodiment, a sense of chord progression in the music is created that is linked to the user's walking speed, creating a sense of unity between the user's walking experience and the experience of the music progressing. [Industrial applicability]

[0210] As a music production support system, it can be used for creating melodies and chord progressions, arranging music, and more.

[0211] As a music playback processing system, it can be used in live performances and everyday life to create effects such as real-time changes in tonality and chord progression.

[0212] It can be used in game development and user experience events for background music playback, including the realization of interactive background music.

[0213] It can be used in educational settings as a visualization tool for the 7CM theory. [Explanation of symbols]

[0214] 101 Host application (DAW, etc.) 11. Personal computers and other electronic computers 111 Storage devices 1111 7CM file 1112 Functional Sound Progression File 1117 MIDI files 112 Keyboard / mouse for connecting to personal computers and other electronic devices 113 Speakers for connecting to personal computers and other electronic computers 114 MIDI devices connected to personal computers and other electronic computers 115. CPU of personal computers and other electronic computers 116 Memory of personal computers and other electronic computers V1 Music Control System

[0215] 201 Input MIDI Information (PolyTouchIn) 202 Input MIDI Information (NoteIn) 212 7CM Change Note Judgment 213 Center tone change note detection 214 Function sound change note detection 215 Performance Mode Change Note Judgment 216 Sequence Operation Note Determination 241 Various information (7CM status, center tone value, function tone value, playback mode, sequence information, etc.) 251 Output MIDI Information (PolyTouchOut) 252 Output MIDI Information (NoteOut) 253 Output MIDI Information (BendOut) C1 center tone designator F1 Function Sound Designation Device G1 Glow Sense Designation Device M1 7CM Display / Designation Device N1 Note Conversion Output Device P1 Performance Mode Designation Device S1 Sequence Designation Device

[0216] 311 7CM File Reading Device 312 7CM File Writing Device 313 Function Sound Progression File Reading Device 314 Function Sound Progression File Writing Device

[0217] 401 Key Input Information from Electronic Computers such as Personal Computers 412 7CM Setting Character Key Judgment 413 Central Tone Setting Character Key Judgment 414 Function Sound Setting Character Key Judgment 415 Performance Mode Setting Character Key Judgment 416 Sequence Setting Character Key Judgment

[0218] G1-1 Glow Sense Change Screen Operation Section G1-101 Slider for Changing Glow Value of Lower Tone Column (1st Degree Type) G1-102 Slider for Changing Glow Value of Lower Chroma (2nd Degree Type) G1-103 Slider for Changing Glow Value of Main Brightness (3rd Degree Type) G1-104 Slider for Changing Glow Value of Upper Luminance (4th Degree Type) G1-105 Slider for Changing Glow Value of Upper Tone Column (5th Degree Type) G1-106 Slider for Changing Glow Value of Main Chroma (6th Degree Type) G1-107 Slider for Changing Glow Value of Main Luminance (7th Degree Type) G1-108 Button for Decreasing Glow Value for Each Glow G1-109 Button for Increasing Glow Value for Each Glow G1-110 Numeric Box for Displaying Glow Value for Each Glow G1-111 Slider for Displaying and Changing Average Glow Value of Main Luminance and Main Chroma G1-112 Sliders for displaying and changing the average brightness of primary saturation and primary brightness. G1-113 Display and change sliders for the average luminance and primary brightness. G1-114 Sliders for displaying and changing the average values ​​of primary brightness, primary saturation, and primary lightness. G1-115 Button for reducing the luminosity value of multiple luminosity effects. G1-116 Button for increasing the luminosity value of multiple luminosity effects G1-121 Priority Tonality Visualization Drawing Frame (Highlights when the displayed tonality is strong)

[0219] M11 7CM display device M11-20 7CM State Drawing Section (Base 7CM of the key major) M11-21 7CM State Drawing Section (Mode-based 7CM with a major key) M11-22 7CM State Drawing Section (H-mode state of the key major) M11-23 7CM State Drawing Section (M / W Conversion State of Key Key Major) M11-30 7CM State Drawing Section (Parallel Key Minor Base 7CM) M11-31 7CM State Drawing Section (Parallel Key Minor Mode System 7CM) M11-32 7CM State Drawing Section (Parallel minor key H-mode state) M11-33 7CM State Drawing Section (Parallel Key Minor Key M / W Conversion State) M11-40 7CM State Drawing Section (Parallel Key Major Bass 7CM) M11-41 7CM State Drawing Section (Parallel Key Major Mode System 7CM) M11-42 7CM State Drawing Section (Parallel Key Major Key H-Shift State) M11-43 7CM State Drawing Section (Parallel Key Major Key M / W Conversion State) M11-10 7CM State Drawing Section (Base 7CM with minor key) M11-11 7CM State Drawing Section (Minor Key Mode System 7CM) M11-12 7CM State Drawing Section (H-mode state of minor key) M11-13 7CM State Drawing Section (M / W conversion state of the key minor key) S1-01 Current sequence block display / designation check box S1-02 Sequence block number display label

[0220] M12 7CM designation device M12-1 Tonality designation part for key (including parallel minor keys) M12-2 Tonality designation part for parallel keys (including parallel major keys) M12-3 Tonality designation part for key major / parallel minor M12-4 Visualization drawing band for priority tonality state (highlight the stronger tonality side of either M12-1 or M12-2) M12-511 7CM (mode system) designation button M12-512 7CM (H / M / W system) designation button M12-521 Modulation button for key parallel minor M12-522 Modulation button for key dominant M12-523 Modulation button for key subdominant M12-524 Modulation button for parallel minor dominant M12-525 Modulation button for parallel minor subdominant M12-526 Modulation button for parallel major M12-527 Modulation button for adjacent 7CM M12-531 7CM file load button M12-532 7CM file load destination path display label M12-533 7CM file write button

[0221] F11 Functional chord progression designation part F11-1 Functional chord progression visualization part F11-11 Functional chord D layer F11-12 Functional chord L layer F11-13 Functional chord S layer F11-14 Functional chord T layer F11-2 Two-dimensional line graph for functional chord definition F11-3 Functional chord progression reflection button F11-4 Knob for central tone designation (only displayed for arbitrary tonality graph) F11-5 Tonality key display label F11-6 Priority Tonality Visualization Drawing Frame (Highlights when the displayed tonality is stronger) S1-03 Sequence Block Key Labels

[0222] F12 Functional Sound Progression Connection Visualization Unit F12-1 1st row function connection status display area F12-10 1st row Function name Function sound display label F12-11 1st stage function connection status (connection to this function) F12-12 1st stage function connection status (connection from this function) F12-13 1st stage function connection status (reserved connection to the function) F12-14 1st stage function connection status (connection held from this function) F12-2 Second row function connection status display area F12-20 2nd row Function name Function sound display label F12-21 Second-stage function connection status (connection to this function) F12-22 Second stage function connection status (connection from this function) F12-23 Second-stage function connection status (reserved connection to the current function) F12-24 Second-stage function connection status (connection held from this function)

[0223] F12-31 Function Sound Progression File Loading Button F12-32 Function sound progression file loading destination path display label F12-33 Function Sound Progression File Writing Button

[0224] 241-11 Central tone key label 241-12 Center tone note section 241-2 7CM display label 241-3 Parallel Tone Status Label P1-1 Performance mode selection button S1-3 Add Block Button S1-4 Block Delete Button

[0225] 202-1 Input MIDI Note Information Visualization Section 202-C1 Input MIDI Note Center Sound Visualization Unit 202-F1 Input MIDI Note Function Sound Visualization Section 202-M1 Input MIDI Note 7CM Visualization Unit 202-N1 Input MIDI Note Conversion Note Visualization Section 202-P1 Input MIDI Note Playback Mode Visualization Section 202-S1 Input MIDI Note Sequence Operation Visualization Unit

[0226] S1-11 Current Block List S1-12 Current Block Previous Button S1-13 Current Block Change Slider S1-14 Current Block Next Button S1-21 Display Block Section S1-22 Previous Display Block Button S1-23 Display Block Change Slider S1-24 Display Block Next Button

[0227] 202a Notes and intervals after white key correction calculation N1-1 White keyboard correction calculation N1-2 Black Key Change Difference Calculation N1-31 Melody Mode Determination N1-32 Melody Mode MIDI Output N1-41 Bass Mode Determination N1-42 Bass Mode MIDI Output N1-51 Harmony Mode Determination N1-52 Harmony Mode MIDI Output N1-6 Standard MIDI Output

[0228] G1-2 Output to output channels defined for each degree of each iris. N1-323 Note calculation corresponding to frequency (Melody mode) Note output to the channel corresponding to the N1-324 degree (Melody mode) N1-327 Adds the rounded luminosity value corresponding to the frequency to the note number (Melody mode). N1-328 Output channel for all frequencies (Melody mode) N1-329 MIDI note output to output channels for all degrees (Melody mode)

[0229] N1-422 Function tone frequency conversion processing (Bass mode) N1-423 Note calculation corresponding to frequency (Bass mode) Note output to the channel corresponding to the N1-424 degree (Bass mode) N1-427 Adds a rounded value of the luminosity corresponding to the degree to the note number (Bass mode). N1-428 Output channel for all frequencies (Bass mode) N1-429 MIDI note output to output channels for all frequencies (Bass mode)

[0230] N1-521 White key reinterpretation processing for harmony N1-522 Function tone frequency conversion processing (Harmony mode) N1-523 Note calculation corresponding to frequency (Harmony mode) Note output to the channel corresponding to the N1-524 degree (Harmony mode) N1-527 Adds a rounded value of the luminosity corresponding to the degree to the note number (Harmony mode). N1-528 Output channel for all frequencies (Harmony mode) N1-529 MIDI note output to output channels for all degrees (Harmony mode)

[0231] G1-21 Output channels defined for each degree of luminosity G1-3 Real-time retention of each luminosity value

[0232] K1-c key layout (for specifying central tone) K1-f key layout (for specifying function sounds) K1-m1 key layout (for mode-based 7CM specification) K1-m2 key arrangement (for H / M / W 7CM designation) K1-p key layout (for specifying performance mode) K1-s key layout (for specifying sequence operations) K1-b key layout (for block addition and deletion operations)

[0233] W1 Wwise and other game sound engines W1-1 Music Playlist W1-11 Music Segment W1-1111 MIDI data for instrument 1 (only once) W1-1117 MIDI data for instrument 1 (7th interval only) W1-1121 MIDI data for instrument 2 (only once) W1-1127 MIDI data for instrument 2 (7th interval only) W1-1131 MIDI data for instrument 3 (only once) W1-1137 MIDI data for instrument 3 (7th interval only) W1-21 Actor-Mixer for one-time pronunciation W1-27 Actor-Mixer for 7th degree pronunciation W1-2712 Actor-Mixer for 7th interval sounding on instrument 2 W1-2713 Actor-Mixer for 7th interval sounding on instrument 3 W1-272 State representing luminance level W1-2721 Luminance status value (High) W1-2722 Luminance status value (Low) W2 Game Engine W2-1 Brightness state transmitted by the game engine

[0234] O1-1 Game Engine O1-111 Sequence Playback Processing O1-112 Sequence Information O1-12 Game Situation Judgment Processing O1-13 Disconnection Function Sound Calculation Processing O1-14 Connection Function Sound Calculation Processing O1-15 7CM Selection Process O1-16 Code Calculation Process O1-17 Next sequence registration process O1-2 Sound Engine

[0235] E1-111 Track data for base sequence in Embodiment 1 E1-112 Track data for bass sound in Embodiment 1 E1-121 Note data for base sequence in Embodiment 1 E1-122 Note data for base pronunciation in Embodiment 1 E1-131 Bass Music Control System (VSTi) in Embodiment 1 E1-132 Output result of the music control system in Embodiment 1 (bass) E1-141 Input / Output Track for Base 1 in Embodiment 1 E1-147 Input / Output Track for Base 7th in Embodiment 1 E1-148 Input / Output Tracks for Base Total Frequency in Embodiment 1 E1-15 Bass sound source (multitimbral) in Embodiment 1 E1-151 Sound source for bass 1st degree in Embodiment 1 (Ch1 of multitimbral) E1-157 Sound source for bass 7th (multitimbral Ch7) in Embodiment 1 E1-158 Bass sound source for all intervals in Embodiment 1 (Ch8 of multitimbral) E1-211 Track data for harmony sequence in Embodiment 1 E1-212 Harmony sound output track data in Embodiment 1 E1-221 Note data for harmony sequence in Embodiment 1 E1-222 Note data for harmony pronunciation in Embodiment 1 E1-231 Harmony Music Control System (VSTi) in Embodiment 1 E1-232 Output result of the music control system in Embodiment 1 (harmony) E1-241 Input / Output Track for Harmony 1st Degree in Embodiment 1 E1-247 Input / Output Tracks for Harmony 7th in Embodiment 1 E1-248 Input / Output Tracks for Harmony Total Degrees in Embodiment 1 E1-251 Sound source for harmonic 1st degree in Embodiment 1 E1-257 Sound source for harmonic seventh in Embodiment 1 E1-258 Sound source for all degrees of harmony in Embodiment 1

[0236] E1-31 Note number in Embodiment 1 E1-32 Section number in Embodiment 1 E1-331 Note specifying sequence number 1 in Embodiment 1 E1-332 Note specifying the central tone 96 (C note) in Embodiment 1 E1-333 Note specifying the performance mode (Bass mode) in Embodiment 1 E1-334 Note specifying the performance mode (Harmony mode) in Embodiment 1 E1-35 Note specifying the instruction to increment the sequence number by 1 in Embodiment 1 E1-36 A note representing a 1st degree (1st degree above the functional tone) in Bass mode in Embodiment 1. E1-37 Notes representing a four-note chord (1st, 3rd, 5th, 7th) in Harmony mode in Embodiment 1

[0237] E1-1231 Functional sound in Embodiment 1, base sound generation data after applying 7CM (before modification) E1-2321 Functional sound in Embodiment 1, Harmony sound generation data after applying 7CM (before modification) E1-411 Functional sound progression file contents in Embodiment 1 (before modification) E1-412 Contents of the 7CM file in Embodiment 1 (before modification)

[0238] E1-1232 Functional sound in Embodiment 1, base sound generation data after applying 7CM (modified) E1-2322 Functional sound in Embodiment 1, Harmony sound generation data after applying 7CM (modified) E1-711 Functional sound progression file contents in Embodiment 1 (after modification) E1-712 Contents of the 7CM file in Embodiment 1 (after modification)

[0239] E1-411 Two-dimensional line graph of the basic tone in Embodiment 1 (before functional sound progression change) E1-421 Two-dimensional line graph of parallel tones in Embodiment 1 (before functional tone progression change) E1-431 Two-dimensional line graph of fourth interval tonality in Embodiment 1 (before functional tone progression change) E1-441 Functional sound connection point 1 in Embodiment 1 (before functional sound progression change) E1-442 Functional sound connection point 2 in Embodiment 1 (before functional sound progression change) E1-443 Functional sound connection point 3 in Embodiment 1 (before functional sound progression change)

[0240] E1-511 Two-dimensional line graph of the basic tone in Embodiment 1 (before functional sound progression change) E1-521 Two-dimensional line graph of parallel tones in Embodiment 1 (after functional tone progression modification) E1-531 Two-dimensional line graph of fourth interval tonality in Embodiment 1 (after functional tone progression modification) E1-541 Functional sound connection point 1 in Embodiment 1 (after functional sound progression change) E1-542 Functional sound connection point 2 in Embodiment 1 (after functional sound progression change) E1-543 Functional sound connection point 3 in Embodiment 1 (after functional sound progression change) E1-544 Functional sound connection point 4 in Embodiment 1 (after functional sound progression change) E1-545 Functional sound connection point 5 in Embodiment 1 (after functional sound progression change) E1-546 Functional sound connection point 6 in Embodiment 1 (after functional sound progression change) E1-547 Functional sound connection point 7 in Embodiment 1 (after functional sound progression change)

[0241] E1-62 Visualization state after 7CM modification in Embodiment 1

[0242] 30-1 Pre-modification chord progression in an example of the effect of the invention 30-2 Close 7CM row in example of the effect of the invention 30-3 Modified chord progression in an example of the effect of the invention

Claims

1. A music control system characterized by having means to manage and control musical characteristics (at least one of the following: brilliance, 7CM, 7CM structure, relationships between 7CMs, function of functional tones, attraction between functional tones, and the sense of anticipation of chord progressions) based on 7CM theory, and supporting the creation or editing of original music without requiring specialized knowledge of music theory.

2. The music control system according to claim 1, comprising means for manipulating and managing the luminosity parameters of the tonality of the musical space, and characterized in that it dynamically holds and sets the luminosity information according to the operation, input information, or contents of an external file.

3. The music control system according to claim 1 or 2, characterized in that it includes means for visualizing the changing structure within a single key and the relationships between closely overlapping single keys, and for suggesting tonal states, tonal changes, chord borrowing, or modulation opportunities.

4. The music control system according to any one of claims 1 to 3, characterized in that it allows arbitrary selection of a 7CM, which is a single tonality described in the 7CM theory, based on operation, input information, or the contents of an external file, and manages this sequentially according to user specification or the contents of an external file, and also includes means for calculating luminosity parameters according to the specified 7CM.

5. The music control system according to any one of claims 1 to 4, characterized in that it sequentially manages a sequence of functional sounds according to operation, input information, or the contents of an external file, and includes a display unit that visualizes the in-key functions of the functional sounds in a two-dimensional graph, and displays the progression status, chord borrowing, or modulation opportunities in multiple keys in real time.

6. The music control system according to any one of claims 1 to 5, further comprising means for visualizing the attractive and sustained attractive forces between functional sounds based on the connection state between one or two sequentially managed functional sounds, and visualizing the sense of progression expectation between functional sounds in a chord progression.

7. The music control system according to any one of claims 1 to 6, characterized in that the music control system includes means for processing input MIDI note information and outputting music information and pitch bend information from a plurality of channels.

8. The music control system according to any one of claims 1 to 7, characterized in that the music control system includes means for processing input MIDI note information and easily creating music for each of the various musical component parts such as melody, harmony, and bass.

9. The music control system according to any one of claims 1 to 8, characterized in that it sets and manages central tone information, functional tone information, 7CM information, luminosity value information, performance mode, and sequence specification information in accordance with the input MIDI note information.

10. The music control system according to any one of claims 1 to 9, characterized in that the music control system enables the management and reuse of information on tonal units and functional tone units based on the 7CM theory in a file format.

11. The music control system according to any one of claims 1 to 10, characterized in that the music control system can easily perform modulation by UI operation for tonalities that it calculates to be close in tonality according to the 7CM theory.

12. A music control method using the music control system described in claim 1, characterized in that it performs complex melody conversion or chord conversion based on 7CM or luminosity values ​​selected by the user or interactively calculated based on inputs such as user behavior or environmental changes.

13. A music control method using the music control system described in claim 1, characterized in that a complex chord progression arrangement is performed using a functional sound selected by the user or interactively calculated based on inputs such as user behavior or environmental changes.

Images