Intelligent musical instrument, visual expression control method thereof, musical instrument system and storage medium

Abstract

The application discloses an intelligent musical instrument, a visual expression control method thereof, a musical instrument system and a storage medium. The intelligent musical instrument is integrated with a display module. The visual expression control method of the intelligent musical instrument comprises the following steps: label data corresponding to a current performance operation is obtained by analyzing an intelligent score file, wherein the label data comprises at least one of a BPM label, a style label and a rhythm pattern label; animation visual information is determined according to the label data; and a driving control signal is generated according to the animation visual information, so as to drive the display module to display corresponding animation content in real time. According to the technical scheme, the audio-visual fusion expression effect of the intelligent musical instrument is effectively improved, and a better audio-visual fusion experience is brought to the audience.

Description

Technical Field

[0001] This application relates to the field of intelligent musical instrument technology, and in particular to an intelligent musical instrument and its visual expression control method, musical instrument system and storage medium. Background Technology

[0002] In the current field of music performance and instrument design, instruments primarily focus on sound production, sound quality optimization, and performance technique enhancement, with their function and design long centered on auditory experience. However, with the development of multimedia art and cross-media performance forms, audiences' expectations for live performances are no longer limited to auditory enjoyment; they increasingly seek immersive experiences that integrate audiovisual elements.

[0003] To achieve this experience, existing technologies primarily employ two approaches: one involves using external auxiliary systems for visual expression. This involves setting up stage lighting, projection equipment, or computer visualization systems independently of the instruments, with professionals manually or through timecode triggering the visual program based on the musical progression. While this separate approach can produce rich visual effects, it has significant drawbacks: the visual system and the instruments are physically isolated from each other in terms of both physical entity and control logic, resulting in a substantial transmission and processing delay between music and visuals, making precise synchronization difficult. Furthermore, the system's setup and operation require expensive equipment and specialized lighting programming knowledge, making it difficult for ordinary performers to master, significantly raising the barrier to entry for integrated audio-visual performances. The other approach utilizes a few electronic or digital instruments, adding indicator lights and other light-emitting devices for simple lighting displays, such as breathing lights that change with volume or fixed-pattern lighting effects. While this approach physically integrates the light source with the instrument, its control logic is extremely simple, responding only to the most basic physical signals (such as volume). It fails to understand the deeper semantic information of the music, such as style and rhythm, resulting in monotonous and superficial visual effects that lack artistic connection to the music's emotion and structure. Summary of the Invention

[0004] This application provides a visual expression control method for intelligent musical instruments, which aims to enable intelligent musical instruments to independently achieve audiovisual fusion expression, lower the threshold for audiovisual fusion performance, and improve the audiovisual fusion expression effect of intelligent musical instruments, thereby bringing a better audiovisual fusion experience to the audience.

[0005] To achieve the above objectives, this application proposes a visual expression control method for an intelligent musical instrument, wherein the intelligent musical instrument integrates a display module, and the visual expression control method for the intelligent musical instrument includes: The label data corresponding to the current performance operation is obtained by parsing from the intelligent score file. The label data includes at least one of BPM label, style label, and rhythm label. The intelligent score is a structured digital file that adds BPM label, style label, and rhythm label indexed by measure / fragment to the standard note information as a data source for visual generation. The animation visual information is determined based on the label data; Based on the animated visual information, a drive control signal is generated to drive the display module to display the corresponding animated content in real time.

[0006] In some embodiments, the tag data includes BPM tags, and the animation visual information includes animation parameters; determining the animation visual information based on the tag data includes: determining the animation parameters based on the BPM tags.

[0007] In some embodiments, the tag data includes the style tag, and the animation visual information includes animation scene content; determining the animation visual information based on the tag data includes: determining the animation scene content based on the style tag.

[0008] In some embodiments, the tag data includes the rhythmic tag, and the animation visual information includes animated dynamic content; determining the animation visual information based on the tag data includes: determining the corresponding animated dynamic content based on the rhythmic tag.

[0009] In some embodiments, the animation visual information includes animation scene content, animation dynamic content, and animation parameters; the tag data includes the BPM tag, the style tag, and the rhythm tag; determining the animation visual information based on the tag data includes: The corresponding animation scene content is determined based on the style tags; The corresponding animation content is determined based on the rhythm type label; The animation parameters are determined based on the BPM label.

[0010] In some embodiments, determining the corresponding animation scene content based on the style tag includes: The visual rules corresponding to the style tags are called from the basic visual rule set to determine the animation scene content, which includes the primary color, secondary color, basic animation texture, and shape.

[0011] In some embodiments, determining the corresponding animation dynamic content based on the rhythm tag includes: Determine the animated dynamic content in the dynamic database that corresponds to the rhythm type tag; Alternatively, a pre-trained dynamic content generation model can be invoked to generate animated dynamic content based on the rhythmic tags.

[0012] In some embodiments, determining the corresponding animation dynamic content based on the rhythm tag includes: From the dynamic database, determine the animated dynamic content that corresponds to the rhythmic tag and the playing key of the current performance operation; Alternatively, a pre-trained dynamic content generation model can be invoked to generate animated dynamic content based on the rhythmic label and the playing key of the current performance operation.

[0013] In some embodiments, determining the animation parameters based on the BPM tag includes: According to the formula V=BPM / 60 n is the animation playback speed; where V is the animation playback speed and n is the set magnification factor. Alternatively, calculate the duration of each beat based on the BPM tag, and determine the duration of each beat as the playback duration of each frame of the animation.

[0014] In some embodiments, determining the animation visual information based on the tag data further includes: After determining that the BPM label corresponding to the current performance operation has changed, the animation parameters of the animation visual information are updated based on the changed BPM label when the current music beat ends.

[0015] In some embodiments, determining the animation visual information based on the tag data further includes: When the determined animation content is a user-defined animation material, the user-defined animation material is subjected to time remapping processing according to the current BPM tag, so that its total playback duration matches the total duration of the beat calculated based on the current BPM tag.

[0016] In some embodiments, determining the animation visual information based on the tag data further includes: When the determined animation scene content is a user-defined static image material, a corresponding color filter is applied to the user-defined static image material according to the current style tag.

[0017] In some embodiments, the step of parsing and obtaining the tag data corresponding to the current performance operation from the smart score file includes: The first music segment in the intelligent score corresponding to the current performance operation is determined, and tag data is obtained from the first music segment, wherein each music segment in the intelligent score has tag data.

[0018] In some embodiments, the step of parsing and obtaining the tag data corresponding to the current performance operation from the smart score file is performed when the performance operation is executed after the smart score has been determined. The visual expression control method for the intelligent musical instrument further includes: when performing a performance operation without determining the intelligent score, obtaining the current BPM label and / or style label of the intelligent musical instrument, and / or obtaining the current style label of the intelligent musical instrument, and determining the rhythm label based on the current style label and the performance key of the current performance operation.

[0019] This application proposes an intelligent musical instrument, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of the visual expression control method of the intelligent musical instrument described in any of the above embodiments.

[0020] This application also proposes a musical instrument system, which includes a mobile terminal with a musical instrument APP installed and a smart musical instrument as described above. The musical instrument APP is used to communicate and send smart sheet music to the smart musical instrument.

[0021] In some embodiments, the musical instrument app is also used to allow users to edit the tag data of the intelligent sheet music, and to update the edited intelligent sheet music to the intelligent musical instrument and / or save it to the sheet music database; and / or, The musical instrument app is also used to allow users to upload visual rules to the smart musical instrument to expand or cover the basic visual rule set corresponding to style tags; and / or, The musical instrument app is also used to allow users to upload custom materials and bind them to application rules, which include associating the custom materials with specific style tags or rhythm tags.

[0022] This application also proposes a storage medium storing a computer program, which, when executed by a processor, implements the steps of the visual expression control method for the intelligent musical instrument described in any of the above embodiments.

[0023] The technical solution of the visual expression control method for intelligent musical instruments in this application involves parsing and obtaining the tag data corresponding to the user's current playing operation from the intelligent music score file when the user plays the intelligent musical instrument. This is equivalent to obtaining the corresponding musical characteristics of the musical sound produced by the current playing operation. Then, based on the obtained tag data, the animation visual information that matches the musical sound produced by the current playing operation is determined. Finally, a drive control signal is generated based on the animation visual information to drive the display module to display the corresponding animation content in real time. In this way, when a user plays a smart instrument, for each playing action, an animated content matching the music sound triggered by that action is displayed in real time on the display module. This ensures that the animation content displayed on the module is more closely matched to the music throughout the performance, enabling the smart instrument to independently achieve audiovisual fusion expression and lowering the barrier to entry for audiovisual fusion performances. Furthermore, it avoids the transmission and processing delays caused by the physical separation of the visual system and the instrument in terms of control logic, allowing for more precise synchronization between vision and music. This effectively enhances the visual and musical fusion expression of the smart instrument, more fully reflecting the emotional changes in the music and providing the audience with a better audiovisual fusion experience. Attached Figure Description

[0024] Figure 1 This is a flowchart illustrating an embodiment of the visual expression control method for an intelligent musical instrument according to this application; Figure 2 This is a flowchart illustrating an embodiment of the visual expression control method for an intelligent musical instrument according to this application; Figure 3 This is a flowchart illustrating an embodiment of the visual expression control method for an intelligent musical instrument according to this application; Figure 4 This is a flowchart illustrating an embodiment of the visual expression control method for an intelligent musical instrument according to this application; Figure 5 This is a flowchart illustrating an embodiment of the visual expression control method for an intelligent musical instrument according to this application; Figure 6 This is a schematic diagram of the structure of an intelligent musical instrument in the hardware operating environment involved in the embodiments of this application. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0026] It should be noted that all directional indicators (such as up, down, left, right, front, back, etc.) in the embodiments of this application are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicator will also change accordingly.

[0027] It should also be noted that when a component is described as "fixed to" or "set on" another component, it can be directly on the other component or there may be an intervening component present. When a component is described as "connected to" another component, it can be directly connected to the other component or there may be an intervening component present.

[0028] Furthermore, the use of terms such as "first" and "second" in this application is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. Additionally, the technical solutions of the various embodiments can be combined with each other, but only on the basis of being achievable by those skilled in the art. When the combination of technical solutions is contradictory or impossible to implement, such a combination of technical solutions should be considered non-existent and not within the scope of protection claimed in this application.

[0029] With the development of multimedia art and cross-media performance forms, audiences' expectations for live performances have expanded beyond auditory enjoyment to include immersive audiovisual experiences. To achieve this, existing technologies primarily employ two approaches: one involves using external auxiliary systems for visual expression, i.e., setting up stage lighting, projection equipment, or computer visualization systems independently of the instruments, with professionals manually or through timecode triggering the visual program based on the musical progression. While this separate approach can produce rich visual effects, it has significant drawbacks: the visual system and the instruments are physically isolated from each other in terms of both physical entity and control logic, resulting in a considerable transmission and processing delay between music and visuals, making precise synchronization difficult; furthermore, the setup and operation of such systems require expensive equipment and specialized lighting programming knowledge, making them difficult for ordinary performers to master, significantly raising the barrier to entry for audiovisual fusion performances.

[0030] Another approach involves adding indicator lights and other light-emitting devices to a few electronic or digital musical instruments for simple lighting displays, such as breathing lights that change with volume or fixed lighting effects. Although this approach physically integrates the light source with the instrument, its control logic is extremely simple, only responding to the most basic physical signals (such as volume). It cannot understand the deeper semantic information of the music, such as style and rhythm, resulting in monotonous and superficial visual effects that lack artistic relevance to the emotions and structure of the music.

[0031] In view of the above-mentioned shortcomings of existing electronic musical instruments, the inventors of this application propose a solution, namely, a visual expression control method for intelligent musical instruments, which enables intelligent musical instruments to independently achieve audiovisual fusion expression, improves the synchronization between vision and music, effectively enhances the visual and musical fusion expression of intelligent musical instruments, more fully reflects the emotional changes in music, and thus brings a better audiovisual fusion experience to the audience.

[0032] The smart musical instrument of this application integrates a display module, which may include a display screen and / or an LED matrix. The smart musical instrument of this application may be a smart guitar, a smart piano, or other similar instrument.

[0033] The visual expression control method for intelligent musical instruments described in this application is implemented using an intelligent musical instrument as an example. Of course, the implementation subject of this method can also be an electronic device connected to the intelligent musical instrument via wired or wireless communication.

[0034] Reference Figure 1 In this embodiment, the visual expression control method for intelligent musical instruments includes: Step S10: Parse and obtain the tag data corresponding to the current performance operation from the intelligent score file.

[0035] When a user plays a smart musical instrument, each performance action (such as triggering chord keys or function keys) will cause the instrument to produce a corresponding performance response (such as generating chord tones, single notes, melodies, or preset sound effects). Upon detecting the user's performance action, the smart instrument acquires the tag data corresponding to that action. This tag data includes at least one of the following: BPM (Beat Per Minute) tag, style tag, and rhythm tag. These tags all carry corresponding musical semantics; that is, they each reflect specific musical characteristics. Therefore, acquiring the tag data corresponding to the current performance action is equivalent to acquiring one or more corresponding musical features of the musical sound produced by that action.

[0036] When the smart musical instrument of this application is played by a user, there are two scenarios: one is that the user selects a smart score and plays according to the smart score; the other is that the user does not select a smart score and plays as they please.

[0037] In this application, the intelligent musical score is a structured digital file that, based on standard note information, adds BPM tags, style tags, and rhythmic tags indexed by measure / musical fragment as a data source for visual generation; that is, each musical fragment in the intelligent musical score has corresponding tag data. The tag data for each musical fragment can include one or more of the following: BPM tag, style tag, and rhythmic tag.

[0038] Therefore, in this embodiment, when performing a performance operation with a determined intelligent score, the intelligent musical instrument can parse and obtain the tag data corresponding to the current performance operation from the intelligent score file, which may include: determining the first music segment in the intelligent score that corresponds to the current performance operation, and obtaining the tag data from the first music segment.

[0039] The intelligent score is essentially composed of several sequentially arranged musical segments. When a user plays the intelligent instrument according to the score, the score progresses sequentially with each performance action. Therefore, each performance action corresponds to a musical segment at a specific point in the score. The intelligent instrument can then determine the corresponding musical segment (the first segment) based on the user's current performance action and retrieve its tag data, thus obtaining the tag data corresponding to the current performance action. Additionally, the intelligent instrument can generate a measure pointer synchronized with the performance. When the user correctly triggers a performance action, the intelligent instrument retrieves the tag data of the musical segment or measure where the current measure pointer is located, i.e., the tag data corresponding to the current performance action.

[0040] Of course, if the user's current playing operation is incorrect (such as pressing the wrong chord key), the step of obtaining the tag data corresponding to the current playing operation can be skipped if the smart score is selected.

[0041] In some embodiments, the smart musical instrument itself has BPM adjustment and style storage functions. That is, the smart musical instrument currently has a BPM tag and a style tag set. There are multiple BPM tags and style tags for the smart musical instrument, and users can manually switch or adjust them. Therefore, when a user performs a performance operation without knowing the smart sheet music, the smart musical instrument obtains the tag data corresponding to the current performance operation. This may include: obtaining the smart musical instrument's current BPM tag and / or style tag; and / or, obtaining the smart musical instrument's current style tag, and determining the rhythmic tag based on the current style tag and the key presses of the current performance operation. The style tag consists of one or more measures of rhythmic patterns, and its style is determined by the combination of rhythmic patterns. Therefore, by determining the current style tag and the current key presses, the rhythmic tag corresponding to the current performance operation can be determined.

[0042] Of course, in other embodiments, the intelligent musical instrument may acquire the tag data corresponding to the current playing operation using other means.

[0043] Step S20: Determine the animation visual information based on the label data.

[0044] The intelligent musical instrument can store a pre-built visual rule mapping engine that maps animation visual information to tag data. After acquiring the tag data corresponding to the current performance operation, the intelligent musical instrument determines the corresponding animation visual information based on the visual rule mapping engine. Furthermore, in some embodiments, the intelligent musical instrument can also invoke a visual recommendation model (which can be an AI intelligent model or a pre-trained recommendation model), inputting the acquired tag data into the AI ​​visual recommendation model. The AI ​​visual recommendation model then integrates the features of the intelligent musical score with the current tag data to determine the animation visual information that best matches the sound produced by the current performance operation.

[0045] Step S30: Generate a drive control signal based on the animation visual information to drive the display module to display the corresponding animation content in real time.

[0046] After determining the animated visual information, the intelligent musical instrument analyzes the animated visual information to generate corresponding drive control signals. These drive control signals (such as PWM data signals) drive the display module, enabling the display module to display the corresponding animated content (that is, the animated content corresponding to the animated visual information) in real time.

[0047] The technical solution of the visual expression control method for intelligent musical instruments in this embodiment involves parsing and obtaining the tag data corresponding to the user's current playing operation from the intelligent music score file when the user plays the intelligent musical instrument. This is equivalent to obtaining the corresponding musical characteristics of the music sound generated by the current playing operation. Then, based on the obtained tag data, the animation visual information that matches the music sound generated by the current playing operation is determined. Finally, a drive control signal is generated based on the animation visual information to drive the display module to display the corresponding animation content in real time. In this way, when a user plays a smart instrument, for each playing action, an animated content matching the music sound triggered by that action is displayed in real time on the display module. This ensures that the animation content displayed on the module is more closely matched to the music throughout the performance, enabling the smart instrument to independently achieve audiovisual fusion expression and lowering the barrier to entry for audiovisual fusion performances. Furthermore, it avoids the transmission and processing delays caused by the physical separation of the visual system and the instrument in terms of control logic, allowing for more precise synchronization between vision and music. This effectively enhances the visual and musical fusion expression of the smart instrument, more fully reflecting the emotional changes in the music and providing the audience with a better audiovisual fusion experience.

[0048] In some embodiments, the tag data includes BPM tags, i.e., BPM values ​​(e.g., 90, 100, 120, 130); the animation visual information includes animation parameters, which may include the playback speed of the animation or the playback duration of each frame of the animation, etc.

[0049] In this embodiment, determining the animation visual information based on the tag data (i.e., step S20) includes: determining animation parameters based on the BPM tag. The intelligent musical instrument determines the animation parameters based on the BPM tag, which means calculating the animation parameters for the visual animation information based on the BPM value. When the tag data only includes the BPM tag, in addition to determining the animation parameters based on the BPM tag, the intelligent musical instrument may also determine the animation scene content and / or animation dynamic content. After determining the animation scene content and / or animation dynamic content, the determined animation scene content and / or animation dynamic content are adjusted according to the animation parameters to obtain the visual animation information. In this embodiment, the intelligent musical instrument determines the animation scene content and / or animation dynamic content by randomly selecting animation scene content and / or animation dynamic content from the database, or by using default animation scene content and / or animation dynamic content, etc.

[0050] In this embodiment, the intelligent musical instrument calculates and determines animation parameters based on the BPM tag corresponding to the current performance operation. This determines the playback speed of the animation or the playback duration of each frame. In this way, the animation parameters of the determined visual animation information can match the music speed generated by the current performance operation. As a result, the animation content displayed by the display module in real time can be dynamically synchronized with the music, achieving a greater unity between visual effects and the emotional expression of the music. This enhances the fusion of visual and musical expression of the intelligent musical instrument and brings a better audiovisual experience to the audience.

[0051] In some embodiments, the tag data includes style tags, and the animation visual information includes animation scene content; wherein, the animation scene content may be an animation background image, such as a landscape background image, a cyberpunk city background image, a space background image, a floral background image, an ink painting background image, etc. In this embodiment, determining the animation visual information based on the tag data (i.e., step S20) includes: determining the animation scene content based on the style tags.

[0052] The intelligent musical instrument determines the animation scene content based on style tags. This can be done by directly retrieving the corresponding animation scene content from a scene database (local or online) based on the mapping relationship between style tags and animation scene content. The animation scene content can include: primary color, secondary color, basic animation texture, and shape. Alternatively, the intelligent musical instrument can determine the animation scene content based on the mapping relationship between style tags and visual rules, determine the corresponding visual rules, and then generate the animation scene content according to the visual rules. In this way, the intelligent musical instrument can obtain animation scene content that closely matches the style or genre of the music generated by the current performance operation based on the style tags in the tag data.

[0053] It should be noted that, in addition to the animation scene content, the animation visual information in this embodiment may also include animation parameters and / or animation dynamic content. When the tag data only includes style tags, the intelligent musical instrument may determine the animation scene content based on the style tags, and may also determine the animation dynamic content and / or animation parameters. Specifically, the intelligent musical instrument may determine the animation dynamic content by randomly selecting it from a database, or by using default or preset animation dynamic content; and the intelligent musical instrument may determine the animation parameters by using default animation parameters (such as a preset playback speed or a preset single-frame playback duration).

[0054] In this embodiment, the intelligent musical instrument determines the corresponding animation scene content based on the style tag corresponding to the current playing operation, so as to obtain animation scene content that matches the style of the music generated by the current playing operation. In this way, the style of the animation content displayed by the display module in real time is more consistent with or better matches the music style, realizing a more unified visual effect and the emotional expression of the music sound, promoting the fusion of visual and musical expression of the intelligent musical instrument, and bringing a better audiovisual fusion experience to the audience.

[0055] In some embodiments, the tag data includes rhythmic tags, and the animation visual information includes animated dynamic content; wherein, the animated dynamic content may be animation effects, such as dance moves, speeding sports cars, or exploding fireworks. In this embodiment, determining the animation visual information based on the tag data (i.e., step S20) includes: determining the corresponding animated dynamic content based on the rhythmic tags.

[0056] In this process, the intelligent musical instrument determines the animation content based on rhythmic tags. This can be done by directly retrieving the corresponding animation content from an animation database (local or online) based on the mapping relationship between rhythmic tags and animation content. Alternatively, the intelligent musical instrument can determine the animation content based on rhythmic tags by feeding the rhythmic tags into a pre-trained animation content generation model. This model can generate animation content that matches the current rhythmic tag by combining the current rhythmic tag with the rhythmic tags and animation content of previous performance operations.

[0057] It should be noted that, in addition to the animated dynamic content, the animated visual information in this embodiment may also include animation parameters and / or animated scene content. When the tag data only includes rhythm tags, the intelligent musical instrument may determine the animated dynamic content and / or animation scene content and / or animation parameters in addition to determining the animated dynamic content based on the rhythm tags. Specifically, the intelligent musical instrument may determine the animated scene content by randomly selecting it from a database or by using default or preset animated scene content; and the intelligent musical instrument may determine the animation parameters by using default animation parameters (such as preset playback speed or preset single-frame playback duration).

[0058] In this embodiment, the intelligent musical instrument determines the corresponding animation content based on the rhythmic tag of the current playing operation, so as to obtain animation scene content that better matches the rhythm of the music generated by the current playing operation. In this way, the changes in the animation content displayed by the display module in real time are more consistent with the changes in the rhythm of the music, achieving a greater unity between visual effects and the emotional expression of the music, promoting the fusion of visual and musical expression of the intelligent musical instrument, and bringing a better audiovisual fusion experience to the audience.

[0059] In some embodiments, the animation visual information includes animation scene content, animation dynamic content, and animation parameters, and the tag data includes BPM tags, style tags, and rhythm tags. In this embodiment, determining the animation visual information based on the tag data (i.e., step S20) includes: Step S21: Determine the corresponding animation scene content based on the style tags; Step S22: Determine the corresponding animation dynamic content based on the rhythm tag; Step S23: Determine the animation parameters based on the BPM tag.

[0060] This embodiment combines the solutions of the three embodiments described above. Specifically, in this embodiment, the intelligent musical instrument determines the animation scene content based on style tags, the corresponding animation dynamic content based on rhythm tags, and the animation parameters based on BPM tags.

[0061] Therefore, the solution in this embodiment makes the animation scene content determined by the intelligent musical instrument more suitable for the style or genre of the music generated by the current performance operation, the determined animation dynamic content better matches the rhythm of the music generated by the current performance operation, and the determined animation parameters match the tempo of the music generated by the current performance operation. In other words, the animation visual information determined by the intelligent musical instrument based on the tag data is compatible with the music generated by the current performance operation in terms of style, rhythm, and dynamic synchronization. Furthermore, the intelligent musical instrument generates drive control signals based on the animation visual information to drive the animation content displayed in real time by the display module. This ensures that the animation content displayed by the display module and the music generated by the current performance operation are compatible in terms of style, tone, special effects rhythm, and dynamic synchronization. This means that the intelligent musical instrument can automatically generate and display matching animation visual effects based on the high-level characteristics of the music being played. This allows the animation content displayed by the display module to unify the emotional expression of the music played by the intelligent musical instrument and has a mutually reinforcing effect, effectively enhancing the fusion of the intelligent musical instrument's visuals and music, and providing the audience with a more immersive audiovisual experience.

[0062] In some embodiments, determining the corresponding animation scene content based on the style tag (i.e., step S21) includes: calling the visual rule corresponding to the style tag from the basic visual rule set to determine the animation scene content.

[0063] The intelligent musical instrument and / or the cloud have a basic visual rule set, which stores the visual rules corresponding to each style tag. When the intelligent musical instrument obtains the style tag corresponding to the current performance operation, it calls the visual rule corresponding to the current style tag from the basic visual rule set according to the mapping relationship between visual rules and style tags. Then, it defines the main color, auxiliary color, basic animation texture and shape according to the visual rule, and generates a defined animation scene content. The animation scene content includes the defined main color, auxiliary color, basic animation texture and shape.

[0064] In this embodiment, the corresponding visual rules are called according to the style tag to define various feature information (primary color, secondary color, basic animation texture and shape) in the animation scene content. The animation scene content obtained in this way is more compatible with the style tone of the style tag. In this way, the animation scene content displayed by the display module in real time is consistent with the style tone of the music being played. The visual scene can better blend with the music to express emotions and enhance the audience's audiovisual experience.

[0065] In some embodiments, determining the corresponding animation dynamic content based on the rhythm tag (i.e., step S22) includes: determining the animation dynamic content corresponding to the rhythm tag in the dynamic database.

[0066] The intelligent musical instrument can store a dynamic database locally and / or on a cloud server. This dynamic database stores animated content corresponding to any rhythmic pattern tag; that is, matching animated content is set according to the characteristics of each rhythmic pattern. For example, rhythmic pattern 1 corresponds to animated content A, rhythmic pattern 2 corresponds to animated content B, rhythmic pattern 3 corresponds to animated content C, rhythmic pattern 4 corresponds to animated content D, and so on. Each animated content (A, B, C, D) can have different animation actions. After obtaining the tag data corresponding to the current playing operation, the intelligent musical instrument retrieves the animated content corresponding to that rhythmic pattern tag from the dynamic database based on the obtained tag data.

[0067] In this embodiment, the dynamic data includes pre-set animated content corresponding to various rhythmic tags. This allows for the real-time display of animated content that matches the music being played on the display module for each performance operation, ensuring precise adaptation and synchronization between the animated content and the music. This results in a unified audiovisual fusion expression effect and enhances the overall audiovisual fusion effect.

[0068] In some embodiments, determining the corresponding animated dynamic content based on the rhythm tag (i.e., step S22) includes: calling a pre-trained dynamic content generation model to generate animated dynamic content based on the rhythm tag.

[0069] The intelligent musical instrument or cloud server has a pre-trained dynamic content generation model. After obtaining the tag data corresponding to the current performance operation, the intelligent musical instrument calls the dynamic content generation model and inputs the rhythm pattern tag from the tag data into the dynamic content generation model. The dynamic content generation model then generates corresponding animated dynamic content based on the input rhythm pattern tag. In this embodiment, the animated dynamic content is generated by the dynamic content generation model according to the rhythm pattern corresponding to the current performance operation, so that the animated dynamic content displayed in real time by the display module accurately matches the music being played, thus unifying the audiovisual fusion expression effect and improving the audiovisual fusion effect.

[0070] In some embodiments, determining the corresponding animation dynamic content based on the rhythm tag includes: determining the animation dynamic content that corresponds to both the rhythm tag and the playing key of the current performance operation from the dynamic database.

[0071] The intelligent musical instrument can store a dynamic database locally, and / or on a cloud server. This dynamic database stores animated content corresponding to any combination of rhythmic tags and any playing key. For example, (rhythm type 1, playing key 1) corresponds to animated content A, (rhythm type 1, playing key 2) corresponds to animated content B, (rhythm type 2, playing key 1) corresponds to animated content C, (rhythm type 2, playing key 2) corresponds to animated content D, and so on. Each animated content (A, B, C, D) can have different animation actions. After acquiring the tag data corresponding to the current playing operation, the intelligent musical instrument, based on the rhythmic tag in the acquired tag data and the playing key of the current playing operation, retrieves the animated content jointly corresponding to the current rhythmic tag and the playing key from the dynamic database according to the mapping relationship between (rhythmic tag, playing key) and animated content.

[0072] In this embodiment, the animated dynamic content is determined by two factors: the rhythmic tag corresponding to the current performance operation and the current key press. This makes the transitions between actions corresponding to adjacent chord keys smoother during performance, making the changes in the animated dynamic content more logical, improving the visual experience, and thus promoting the audiovisual fusion expression effect.

[0073] In some embodiments, determining the corresponding animated dynamic content based on the rhythm pattern label includes: invoking a pre-trained dynamic content generation model to generate animated dynamic content based on the rhythm pattern label and the key presses of the current performance operation. The smart musical instrument or cloud server has a pre-trained dynamic content generation model. After obtaining the label data corresponding to the current performance operation, the smart musical instrument invokes the dynamic content generation model and inputs the rhythm pattern label from the label data and the key press information of the current performance operation into the dynamic content generation model. The dynamic content generation model then generates the corresponding animated dynamic content based on the input information.

[0074] In this embodiment, the animated dynamic content is generated by the dynamic content generation model based on the rhythm pattern and key presses corresponding to the current performance operation. The dynamic content generation model can smoothly connect the dynamic content corresponding to adjacent chord keys, making the changes in the animated dynamic content more logical during performance, improving the visual experience, and thus promoting the audiovisual fusion expression effect.

[0075] In some embodiments, determining the animation parameters based on the BPM label (i.e., step S23) may include: using the formula V=BPM / 60 n is the animation playback speed; where V is the animation playback speed and n is the set magnification factor; that is, in this embodiment, the animation parameter is the animation playback speed. For example, if the BPM is 90 and the magnification factor n=1, then the animation playback speed V=90 / 60. 1 = 1.5.

[0076] In some embodiments, determining animation parameters based on the BPM tag may include: calculating the duration of each beat based on the BPM tag, and determining the duration of each beat as the playback duration of each frame of the animation. That is, in this embodiment, the animation parameter is the playback duration of each frame of the animation. The duration of each beat, T, is calculated as: T = 60 / BPM; for example, when the BPM is 120, the duration of each beat is 60 / 120 = 0.5 seconds, and the playback duration of each frame of the animation is determined to be 0.5 seconds.

[0077] Of course, in other embodiments, the animation parameters can be other parameters, and determining the animation parameters based on the BPM tag can also be done using other methods.

[0078] Reference Figure 3 In some embodiments, determining the animation visual information based on the tag data (i.e., step S20) further includes: Step S24: After determining that the BPM label corresponding to the current performance operation has changed, the animation parameters of the animation visual information are updated based on the changed BPM label when the current music beat ends.

[0079] In this application, the intelligent sheet music is provided to the intelligent musical instrument by a musical instrument app installed on a mobile terminal (such as a smartphone, smartwatch, tablet, etc.) via communication (such as wireless communication via Bluetooth, WIFI, etc., or wired communication). Users can customize and modify the data of the intelligent sheet music through the musical instrument app (such as changing BPM, style, rhythm, etc.). When a user edits and modifies the intelligent sheet music they are playing, the musical instrument app will push the edited intelligent sheet music data to the intelligent musical instrument in real time and differentially for updating.

[0080] When the intelligent musical instrument receives updated smart score data, it plays according to the updated score. If it determines that the BPM tag corresponding to the current performance operation in the new smart score is different from the previous one (i.e., a change has occurred), the intelligent musical instrument will not immediately apply the new BPM tag to recalculate the animation parameters. Instead, it waits until the current musical beat ends before updating the animation parameters of the visual information based on the new BPM tag (i.e., the changed BPM tag), so that the animation content for the next beat is played and displayed according to the new animation parameters. This avoids sudden changes in the playback speed of the animation content within the current musical beat, which would affect the visual effect, and also avoids sudden changes in the tempo of the music within the current musical beat, which would affect the sound effects of the performance, ensuring a smooth transition in the music.

[0081] Reference Figure 4 In some embodiments, determining the animation visual information based on the tag data (i.e., step S20) further includes: Step S25: When the determined animation dynamic content is a user-defined animation material, perform time remapping processing on the user-defined animation material according to the current BPM tag, so that its total playback duration matches the total duration of the beat number calculated based on the current BPM tag.

[0082] The animated content in the smart musical instrument's dynamic database includes not only system-preset animation materials but also user-uploaded custom animation materials via the instrument's app. When users upload animation materials through the app, they select which rhythm tag to bind the animation material to (e.g., if the animation material is selected for use as a display effect during the "chorus burst," it will be bound to the rhythm tag corresponding to "chorus burst"). The system-preset animation materials are all officially uploaded and uniformly processed, and can be used directly. However, user-uploaded animation materials can vary greatly and usually cannot be used directly, requiring appropriate processing.

[0083] In this embodiment, when the intelligent musical instrument determines the corresponding animated content based on the rhythmic pattern label, if it determines that the animated content is user-defined animation material, it performs time remapping processing on the user-defined animation material according to the BPM label corresponding to the current performance operation (i.e., the current BPM label). This remapping process includes adjusting the frame intervals in the animation material and generating intermediate frames through interpolation, so that the total playback duration of the animation material matches the total duration of the time signature calculated based on the current BPM label. This ensures that the playback duration of the remapped animation material is equal to or substantially equal to the duration of the current measure. In this way, the intelligent musical instrument allows any user-uploaded dynamic material to be matched with its corresponding rhythmic pattern, enabling the display module to play user-defined animation materials. This better meets the user's customization needs and allows for personalized expression through user-defined animation materials.

[0084] Reference Figure 5 In some embodiments, determining the animation visual information based on the tag data (i.e., step S20) further includes: Step S26: When the determined animation scene content is a user-defined static image material, apply the corresponding color filter processing to the user-defined static image material according to the current style tag.

[0085] In addition to the system's preset animation scene materials, the smart musical instrument's database also includes user-uploaded custom animation scene materials (such as static image materials) via the instrument's app. When users upload animation scene materials through the app, they select which style tag to bind the animation scene material to (e.g., if a static image is selected for the "Cyberpunk" style, it will be bound to the corresponding "Cyberpunk" style tag). However, the colors of user-uploaded static image materials may not match the style tag selected by the user, and usually require appropriate processing.

[0086] In this embodiment, when the intelligent musical instrument determines the corresponding animation scene content based on the style tag corresponding to the current performance operation (i.e., the current style tag), if it determines that the animation scene content is a user-defined static image, it applies a corresponding color filter to the user-defined static image material according to the current style tag, so that the color scheme of the user-defined static image material matches the style tone of the current style tag. In this way, the intelligent musical instrument allows any static image material uploaded by the user to be matched and used with the style tag selected and bound by the user. This enables the display module to display and play user-defined scene materials, thereby better meeting the user's customization needs and supporting personalized expression.

[0087] It should be noted that, in the absence of any contradictions or conflicts between the above embodiments of the visual expression control method for intelligent musical instruments in this application, the above embodiments can be arbitrarily combined or combined to form new embodiments.

[0088] This application also proposes a smart musical instrument, see [reference] Figure 6 , Figure 6 This is a schematic diagram of the structure of an intelligent musical instrument in the hardware operating environment involved in the embodiments of this application.

[0089] like Figure 6 As shown, the intelligent musical instrument may include: a processor 1001 (e.g., a CPU), a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen and an input unit, such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0090] Those skilled in the art will understand that Figure 6 The intelligent musical instrument structure shown does not constitute a limitation on the intelligent musical instrument and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0091] like Figure 6 As shown, the memory 1005, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and computer programs.

[0092] exist Figure 6 In the intelligent musical instrument shown, the network interface 1004 is mainly used to connect to the backend server and communicate data with the backend server; the user interface 1003 is mainly used to connect to the client (user end) and communicate data with the client; and the processor 1001 can be used to call the computer program stored in the memory 1005. When the computer program is called and executed by the processor 1001, it implements the steps of the above-mentioned visual expression control method of the intelligent musical instrument.

[0093] Based on the computer program proposed in the foregoing embodiments, this application also proposes a storage medium storing a computer program, which, when executed by a controller, implements the visual expression control method for the intelligent musical instrument described in the foregoing embodiments.

[0094] Since both the intelligent musical instrument and the storage medium of this application can implement the steps of the above-described visual expression control method for intelligent musical instruments, they at least have all the beneficial effects brought about by the technical solutions of the above-described embodiments of the visual expression control method for intelligent musical instruments, which will not be elaborated here.

[0095] This application further proposes a musical instrument system, which includes a mobile terminal with a musical instrument app installed and the smart musical instrument described in the above embodiments. Since the musical instrument system uses the aforementioned smart musical instrument, it also possesses at least all the beneficial effects brought about by the technical solutions of the above-described visual expression control method embodiments of the smart musical instrument, which will not be elaborated upon here.

[0096] In this embodiment, the musical instrument app is used to send intelligent sheet music to the smart musical instrument. Specifically, the intelligent sheet music is stored in the musical instrument app's sheet music database or on the mobile terminal where the musical instrument app is installed. When the user plays the smart musical instrument, they first open the musical instrument app on the mobile terminal to establish a communication connection between the mobile terminal and the smart musical instrument (such as via Bluetooth or Wi-Fi). The user then selects the intelligent sheet music to play on the musical instrument app, and the musical instrument app sends the selected intelligent sheet music data to the smart musical instrument.

[0097] In some embodiments, the instrument app also allows users to edit the tag data of the smart score, and to update the edited smart score to the smart instrument and / or save it to the score database. The instrument app's display interface can show the user-selected smart score (including tag data for displayed music segments) in real time. Users can choose to edit the smart score. In edit mode, users can modify the tag data of the smart score. After editing, the instrument app can update the edited smart score to the smart instrument (i.e., send it to the smart instrument) and / or save it to the score database. This allows users to easily and in real-time customize the visual display style of the display module by intuitively editing the tags of the smart score through the instrument app, achieving a "what you see is what you get" visual arrangement and greatly reducing the professional barrier.

[0098] In some embodiments, the Musical Instrument App also allows users to upload visual rules to the Smart Musical Instrument to expand or override the basic visual rule set corresponding to style tags. The Musical Instrument App has the function of customizing, expanding, or modifying visual rules. Users can select visual rules from the system's basic visual rule set and modify the parameter information (such as primary color, secondary color, basic animation texture, or shape) within the visual rules. They can also modify the style tags corresponding to the visual rules. After modification, the modified visual rule data is updated to the Smart Musical Instrument, allowing the Smart Musical Instrument to override the original visual rules with the updated world rules. Users can also add new visual rules by setting the parameter information and corresponding style tags of the new visual rule, thus defining a new visual rule. This new visual rule is then sent to the Smart Musical Instrument, allowing the Smart Musical Instrument to save it to the basic visual rule set. In this way, users can personalize the modification of visual rules, providing a creative window for animation scene content and giving users more creative freedom.

[0099] In some embodiments, the musical instrument app is also used to allow users to upload custom materials and bind them to application rules, which include associating the custom material with a specific style tag or rhythm tag.

[0100] The music instrument app also features a material upload function, allowing users to upload custom animation and image materials. For example, users can upload their own self-created animation or image materials to the database. When uploading materials, users can also select application rules for the uploaded materials within the app, essentially binding application rules to the uploaded materials. For instance, they can associate uploaded custom animation materials with a specific rhythm tag (such as the rhythm tag corresponding to "chorus burst"), and upload custom image materials with a specific style tag (such as the style tag corresponding to "cyberpunk").

[0101] This embodiment supports users uploading and associating custom image materials with various style tags, as well as custom animation materials with various rhythm tags. This allows the display module to play the user's custom materials during performance, enabling personalized expression. This grants users unlimited creative freedom, making the instrument an open audio-visual creation tool, significantly enhancing user engagement and product scalability.

[0102] Furthermore, considering the various embodiments of the above-described visual expression control method for intelligent musical instruments, compared to existing electronic musical instruments or musical instrument systems, the intelligent musical instrument and musical instrument system of this application possess at least the following significant advantages: 1. By embedding the visual generation system as an inherent module into the intelligent musical instrument, sound and light originate from the same performance operation, fundamentally ensuring the unity of sound and image. This creates a highly unified and immersive sensory environment for performers and audiences, greatly expanding the artistic expression dimensions of the instrument.

[0103] 2. It innovatively adopts an "intelligent score tag-driven" mode, using the authoritative BPM and rhythm tags built into the intelligent score as the sole clock source and event trigger for visual generation, avoiding analysis errors and transmission delays from the data source. Visual dynamics achieve precise frame-level synchronization with the musical beat and sections, with response latency down to the millisecond level, meeting the demands of high-level performances.

[0104] 3. Through a three-layer mapping engine (style and tone layer, dynamic synchronization layer, and rhythm effects layer), the system can "understand" the high-level characteristics of music and automatically generate matching visual effects. For example, different musical styles are automatically matched with different colors and textures; different rhythmic patterns trigger specific animation effects. This enables a breakthrough in visual expression, leaping from "simple reaction" to "intelligent interpretation," resulting in a qualitative improvement in expressiveness.

[0105] 4. A low-barrier, open, and personalized creation platform has been built: For ordinary users: By editing smart sheet music tags (such as changing style and rhythm) through the intuitive musical instrument app, you can easily define the visual style and achieve a "what you see is what you get" visual arrangement, which greatly reduces the professional threshold.

[0106] For creative users: The app supports uploading custom images / animations, and through an adaptive material processing module, it can automatically respond to data such as BPM and style (e.g., animation rhythm adapts to BPM, image color tones match musical mood). This grants users unlimited creative freedom, making the instrument an open audio-visual creation tool, significantly improving user engagement and product scalability.

[0107] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between devices or modules, and may be electrical, mechanical, or other forms.

[0108] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0109] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0110] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0111] The above description is only a part or preferred embodiment of this application. Neither the text nor the drawings should limit the scope of protection of this application. All equivalent structural transformations made using the content of this application's specification and drawings under the overall concept of this application, or direct / indirect applications in other related technical fields, are included within the scope of protection of this application.

Claims

1. A visual expression control method for an intelligent musical instrument, characterized in that, The intelligent musical instrument integrates a display module, and the visual expression control method of the intelligent musical instrument includes: The label data corresponding to the current performance operation is obtained by parsing the intelligent score file. The label data includes at least one of BPM label, style label, and rhythm label. The animation visual information is determined based on the label data; Based on the animated visual information, a drive control signal is generated to drive the display module to display the corresponding animated content in real time.

2. The visual expression control method for an intelligent musical instrument according to claim 1, characterized in that, The tag data includes BPM tags, and the animation visual information includes animation parameters; determining the animation visual information based on the tag data includes: determining the animation parameters based on the BPM tags; And / or, The tag data includes the style tags, and the animation visual information includes the animation scene content; determining the animation visual information based on the tag data includes: determining the animation scene content based on the style tags; And / or, The tag data includes the rhythmic tags, and the animation visual information includes the animation dynamic content; determining the animation visual information based on the tag data includes: determining the corresponding animation dynamic content based on the rhythmic tags.

3. The visual expression control method for intelligent musical instruments according to claim 2, characterized in that, The animation visual information includes animation scene content, animation dynamic content, and animation parameters; the tag data includes the BPM tag, the style tag, and the rhythm tag. The step of determining the animation visual information based on the tag data includes: The corresponding animation scene content is determined based on the style tags; The corresponding animation content is determined based on the rhythm type label; The animation parameters are determined based on the BPM label.

4. The visual expression control method for intelligent musical instruments according to claim 2, characterized in that, The step of determining the corresponding animation scene content based on the style tag includes: The visual rules corresponding to the style tags are called from the basic visual rule set to determine the animation scene content, which includes the primary color, secondary color, basic animation texture, and shape.

5. The visual expression control method for an intelligent musical instrument according to claim 2, characterized in that, The step of determining the corresponding animation dynamic content based on the rhythm tag includes: Determine the animated dynamic content in the dynamic database that corresponds to the rhythm type tag; Alternatively, a pre-trained dynamic content generation model can be invoked to generate animated dynamic content based on the rhythmic tags; Alternatively, determining the corresponding animation dynamic content based on the rhythmic tag includes: From the dynamic database, determine the animated dynamic content that corresponds to the rhythmic tag and the playing key of the current performance operation; Alternatively, a pre-trained dynamic content generation model can be invoked to generate animated dynamic content based on the rhythmic label and the playing key of the current performance operation.

6. The visual expression control method for an intelligent musical instrument according to claim 2, characterized in that, The step of determining animation parameters based on the BPM tag includes: According to the formula V=BPM / 60 n is the animation playback speed; where V is the animation playback speed and n is the set magnification factor. Alternatively, calculate the duration of each beat based on the BPM tag, and determine the duration of each beat as the playback duration of each frame of the animation.

7. The visual expression control method for an intelligent musical instrument according to claim 2, characterized in that, The step of determining the animation visual information based on the tag data further includes: After determining that the BPM label corresponding to the current performance operation has changed, the animation parameters of the animation visual information are updated based on the changed BPM label when the current music beat ends.

8. The visual expression control method for an intelligent musical instrument according to claim 2, characterized in that, The step of determining the animation visual information based on the tag data further includes: When the determined animation content is user-defined animation material, the user-defined animation material is subjected to time remapping processing according to the current BPM tag, so that its total playback duration matches the total duration of the beat number calculated based on the current BPM tag. And / or, The step of determining the animation visual information based on the tag data further includes: When the determined animation scene content is a user-defined static image material, a corresponding color filter is applied to the user-defined static image material according to the current style tag.

9. The visual expression control method for an intelligent musical instrument according to any one of claims 1 to 8, characterized in that, The step of parsing and obtaining the tag data corresponding to the current performance operation from the intelligent score file includes: Identify the first music segment in the intelligent score that corresponds to the current performance operation, and obtain tag data from the first music segment, wherein each music segment in the intelligent score has tag data; And / or, the step of parsing and obtaining the tag data corresponding to the current performance operation from the intelligent score file is performed when the performance operation is executed after the intelligent score has been determined; The visual expression control method for the intelligent musical instrument further includes: when performing a performance operation without determining the intelligent score, obtaining the current BPM label and / or style label of the intelligent musical instrument, and / or obtaining the current style label of the intelligent musical instrument, and determining the rhythm label based on the current style label and the performance key of the current performance operation.

10. A smart musical instrument, characterized in that, The intelligent musical instrument includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of the visual expression control method for the intelligent musical instrument as described in any one of claims 1 to 9.

11. A musical instrument system, characterized in that, The musical instrument system includes a mobile terminal with a musical instrument APP installed and the smart musical instrument as described in claim 10, wherein the musical instrument APP is used to communicate and send smart sheet music to the smart musical instrument.

12. The musical instrument system according to claim 11, characterized in that, The musical instrument app is also used to allow users to edit the tag data of the intelligent sheet music, and to update the edited intelligent sheet music to the intelligent musical instrument and / or save it to the sheet music database; and / or, The musical instrument app is also used to allow users to upload visual rules to the smart musical instrument to expand or cover the basic visual rule set corresponding to style tags; and / or, The musical instrument app is also used to allow users to upload custom materials and bind them to application rules, which include associating the custom materials with specific style tags or rhythm tags.

13. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the visual expression control method for an intelligent musical instrument as described in any one of claims 1 to 9.

Images