Information processing method, device, electronic device and storage medium

By obtaining MIDI arrangement templates and music information, and using the automatic arrangement system to generate complete MIDI files, the problems of arrangement complexity and high threshold in the existing technology are solved, and the automatic generation of reasonable arrangement is realized, and music production is improved. efficiency.

CN114005424BActive Publication Date: 2024-12-03BEIJING LINGDONGYIN TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111085242.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-09-16
Publication Date
2024-12-03
Estimated Expiration
2041-09-16

AI Technical Summary

Technical Problem

The prior art has complex arrangement and mixing processes in music production, high threshold, and large training volume of automatic arrangement system, and the generation results lack interpretability and controllability, making it difficult to automatically generate complete and reasonable arrangements.

Method used

By obtaining an arrangement template based on MIDI editing, combining the music information to be arranged, using an automatic arrangement system for data preparation and automatic arrangement processing, a complete MIDI file is generated, simplifying the arrangement process and improving reusability.

Benefits of technology

It lowers the threshold for music production, simplifies the arrangement process, automatically generates complete and reasonable arrangements, improves the efficiency of music production, and solves the problem of arrangement reusability to a certain extent.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114005424B_ABST
    Figure CN114005424B_ABST
Patent Text Reader

Abstract

The present application discloses an information processing method, device, electronic device and storage medium. The specific implementation scheme is: obtaining an arrangement template based on a music file, wherein the music file includes a file edited based on a musical instrument digital interface (MIDI); obtaining music information to be arranged; and obtaining a target object according to the music information and the arrangement template. By using the present application, a complete and reasonable arrangement can be automatically generated.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer music technology, and in particular to an information processing method, device, electronic device and storage medium. Background Art

[0002] Since the information revolution, the way music and multimedia are spread has changed dramatically in a short period of time. This qualitative change has led to an explosive growth in the market demand for all kinds of music: whether it is singles, albums, MVs, karaoke with pop music or artistic creation as the main elements, or short videos, advertisements, animations, promotional films and film and television works that use music as auxiliary, or radio stations, anchors, and public space music that use music as background content, a large amount of original music is needed. How to automatically generate complete and reasonable arrangements has become a technical problem that needs to be solved in this field. Summary of the invention

[0003] The present application provides an information processing method, device, electronic device and storage medium.

[0004] According to one aspect of the present application, there is provided an information processing method, comprising:

[0005] Acquire a music arrangement template based on a music file, wherein the music file includes a file edited based on a musical instrument digital interface (MIDI);

[0006] Get the music information to be arranged;

[0007] A target object is obtained according to the music information and the arrangement template.

[0008] According to another aspect of the present application, there is provided an information processing device, including:

[0009] A template acquisition module, used to acquire an arrangement template based on a music file, wherein the music file includes a file obtained based on MIDI editing;

[0010] An information acquisition module is used to acquire music information to be arranged;

[0011] The synthesis processing module is used to obtain the target object according to the music information and the arrangement template.

[0012] According to another aspect of the present application, there is provided an electronic device, including:

[0013] at least one processor; and

[0014] a memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method provided by any embodiment of the present application.

[0016] According to another aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method provided by any embodiment of the present application.

[0017] The present application obtains an arrangement template based on a music file, wherein the music file includes a file obtained based on MIDI editing; obtains music information to be arranged; and obtains a target object according to the music information and the arrangement template. With the present application, through the arrangement template prepared in advance, the user only needs to input the music information to be arranged (such as simple input of chords, melody and other information) to automatically generate a complete and reasonable arrangement.

[0018] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present application.

[0020] Figure 1 is a flowchart of an information processing method according to an embodiment of the present application;

[0021] Figure 2 is a schematic diagram of a process of automatic music arrangement in an application example according to an embodiment of the present application;

[0022] Figure 3 is a schematic diagram of visualization of a typical MIDI clip in a piano roll according to an application example of an embodiment of the present application;

[0023] Figure 4 According to an application example of an embodiment of the present application, Figure 3 A schematic diagram of the annotation results presented after the annotation;

[0024] Figure 5 is a schematic diagram of a multi-track MIDI project in an application example according to an embodiment of the present application;

[0025] Figure 6 According to an application example of an embodiment of the present application, Figure 5 Schematic diagram of segmentation;

[0026] Figure 7is a schematic diagram of music information to be arranged in an application example according to an embodiment of the present application;

[0027] Figure 8 is a schematic diagram of a process of automatically arranging music by changing an accompaniment to a solo according to an application example of an embodiment of the present application;

[0028] Fig. 9 is a schematic diagram of texture length distribution in an application example according to an embodiment of the present application;

[0029] Fig.10 is a schematic diagram of different effects of looping and stretching from 8 slices to 16 slices when the number of groups is 4 according to an application example of an embodiment of the present application;

[0030] Fig.11 is a schematic diagram of constraint parameters of a keyboard in an application example according to an embodiment of the present application;

[0031] Fig.12 is a schematic diagram of constraint parameters of a guitar in an application example according to an embodiment of the present application;

[0032] Fig.13 is a schematic diagram of chord conversion of a piano performance segment in an application example according to an embodiment of the present application;

[0033] Fig.14 is a schematic diagram of search results for a Cm9 chord in a six-note column chord according to an application example of an embodiment of the present application;

[0034] Fig.15 is a schematic diagram of selecting the best path in a multi-stage dynamic planning stage according to an application example of an embodiment of the present application;

[0035] Fig.16 is a schematic diagram of an accompaniment in an application example according to an embodiment of the present application;

[0036] Fig.17 is a schematic diagram of a solo in an application example according to an embodiment of the present application;

[0037] Fig.18 According to an application example of an embodiment of the present application, t k Schematic diagram of further classification based on hand shape;

[0038] Fig.19 is a schematic diagram of the structure of an information processing device according to an embodiment of the present application;

[0039] Fig. 20 It is a block diagram of an electronic device used to implement the information processing method of an embodiment of the present application. DETAILED DESCRIPTION

[0040] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0041] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The term "at least one" in this article means any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C. The terms "first" and "second" in this article refer to multiple similar technical terms and distinguish them. They do not mean to limit the order or to limit them to only two. For example, the first feature and the second feature refer to two types / two features. The first feature can be one or more, and the second feature can also be one or more.

[0042] In addition, in order to better illustrate the present application, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present application can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present application.

[0043] In related technologies, whether it is traditional audio-visual products such as advertisements, TV series, movies, and games, or emerging short videos and live broadcasts, a large amount of music materials are needed. As for music itself, with the development of various music platforms, more and more people have become musicians and published their works on these music platforms. It can be said that music has become an indispensable part of people's lives.

[0044] At the same time, with the development of recording and production technology, a computer plus a keyboard can make good music. Although there are music production equipment, professional knowledge is still required. In order to make a complete work, in addition to learning music theory to write chords and match chords to melodies, it is also necessary to learn arrangement and mixing. There are already some systems on the market that can realize automatic arrangement. They can be roughly divided into two categories. One is the traditional electronic keyboard, and the other is the emerging deep learning type. Among them, the technical solution of the electronic keyboard is widely used in various performance occasions. It can use pre-made one-key chords. There are also some products with similar principles, such as box sound intelligent music, Band in a Box, etc. This type of product uses the properties of chord scales to map the performance of an instrument under one chord to the playing method of another chord, and then connects them with a large number of music clips to achieve the effect of instrumental writing. The technical solution of the electronic keyboard, roughly speaking, uses the chord scale to transform the playing method of one chord into another chord. The problem is that chords and scales do not correspond one to one. The same chord may correspond to different scales in different situations. Mechanical mapping results in a single result and low originality. As deep learning becomes more popular, some deep learning-related technical solutions have also been applied to music. For example, MuseNet is a model proposed by OpenAI in 2019 that generates multi-track music based on the GPT2 neural network. Through training with a large number of music clips, the neural network can predict the note writing of each instrumentation of multi-track music in a given style. The problem with this deep learning-related technical solution is that the training volume required during the training process is huge, and the generated results lack interpretability and controllability.

[0045] There are hundreds of musical instruments and chords in the world. Each chord can be arranged in many ways on different instruments. Considering the smooth connection between different chord arrangements and the coordination between different instruments, the possibility of arrangement is immeasurable. Mixing requires musicians to have in-depth research on the endless sound sources and mixing plug-ins, and accumulate a lot of experience. This undoubtedly takes a lot of energy from musicians. Compared with the direct first impression given by melody and lyrics, arrangement and mixing do not require so much innovation. In many cases, overly gorgeous arrangements are the main attraction. The threshold for music production is high, but music is needed in large quantities, so a technology that reduces the difficulty of music production is about to emerge.

[0046] In view of the above dilemma, the information processing method proposed in this application is a technical solution for automatic arrangement based on an arrangement template. Considering that on the one hand, arrangement is a necessary step in the music production process, involving many fields and theories, and people need to pay a lot of energy to learn to arrange music; on the other hand, in popular music, the original part is mostly reflected in the melody, lyrics and other aspects of the song, rather than the arrangement, that is, the arrangement has a certain reusability, and there may be a reuse method. The technical solution can make the arrangement template in advance, which is mainly composed of a data preparation module and an automatic arrangement module. The arrangement template can be made and marked by professional musicians. The user only needs to provide music information, such as simply inputting chords, melody and other information. After that, through the calculation of the automatic arrangement module, the arrangement of the arrangement template can be migrated to the new music information, generating a complete and reasonable arrangement, and the music digital file in MIDI format can be exported, thereby not only simplifying the large amount of energy required for the arrangement, automatically generating a complete and reasonable arrangement, but also solving the problem of arrangement reusability to a certain extent, and lowering the threshold of music production.

[0047] According to an embodiment of the present application, an information processing method is provided. Figure 1 It is a flowchart of an information processing method according to an embodiment of the present application. The method can be applied to an information processing device. For example, the device can be deployed in a terminal or a server or other processing device to execute, and can perform arrangement template acquisition, music file editing, etc. The terminal can be a user equipment (UE, User Equipment), a mobile device, a cellular phone, a cordless phone, a personal digital assistant (PDA, Personal Digital Assistant), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementations, the method can also be implemented by a processor calling a computer-readable instruction stored in a memory. For example Figure 1 As shown, including:

[0048] S101. Acquire a music arrangement template based on a music file, wherein the music file includes a file edited based on a musical instrument MIDI.

[0049] S102, obtaining music information to be arranged.

[0050] In one example, the music information to be arranged may include: paragraph, beat signature, tempo, mode, chords, melody and other information.

[0051] S103: Obtain a target object according to the music information and the arrangement template.

[0052] In one example, the target object is a new music file obtained after arrangement processing. For example, the design requirements of the user to customize the new music file can be obtained based on the music information to be arranged, and the material that can be provided by the old MIDI file already in the arrangement template (such as the materials including paragraphs, textures, fragments, slices, track attributes, chords and pitch attributes of each note obtained after the old MIDI file is segmented and annotated) is combined for arrangement processing to obtain the new MIDI file.

[0053] In one example, the arrangement process can be implemented by an automatic arrangement system based on the arrangement template. The automatic arrangement system can be composed of two modules: a data preparation module and an automatic arrangement module. The data preparation module (can be set in the cloud or in the terminal) can obtain the arrangement template according to the above materials.

[0054] The automatic arrangement module is for terminal users and can be set in the terminal. As long as the terminal (electronic device such as a mobile phone or a computer) is installed with the automatic arrangement module (or the corresponding automatic arrangement processing logic), it is within the protection scope of this application. When the terminal user obtains the required arrangement template from the database and inputs the music information to be arranged, a complete digital music file (such as the new MIDI file) can be generated by the automatic arrangement system, and then the new MIDI file is exported.

[0055] By using the present application, it is possible to obtain an arrangement template based on a music file, wherein the music file includes a file obtained based on MIDI editing, obtain the music information to be arranged, and obtain the target object based on the music information and the arrangement template. Thus, through the pre-made arrangement template, the user only needs to input the music information to be arranged (such as simple input of chords, melody and other information) to automatically generate a complete and reasonable arrangement.

[0056] In one embodiment, the method further includes: segmenting and attribute labeling the music file to obtain the arrangement template; wherein the arrangement template includes paragraphs, textures, segments, slices, tracks, and notes. For example, if the music file is the above-mentioned old MIDI file, the segmentation and attribute labeling is to disassemble the above-mentioned old MIDI file to obtain various materials including paragraphs, textures, segments, slices, tracks, chords, and each note pitch.

[0057] In one embodiment, the target object is obtained according to the music information and the arrangement template, including: structurally reorganizing the various materials contained in the arrangement template and re-adjusting the pitch of the notes according to the music information, so as to achieve arrangement processing and obtain the target object. For example, in the case where the music file is the above-mentioned old MIDI file, the arrangement processing is based on the reorganization of the various materials obtained by the above-mentioned segmentation processing and attribute annotation, and re-adjusting the pitch attribute of each note according to the above-mentioned music information to be arranged, so as to obtain the target object (a new MIDI file obtained after the arrangement processing).

[0058] In one embodiment, it also includes: obtaining the paragraphs by segmenting the music file; each of the paragraphs contains at least one of the textures; the length of the segment in each paragraph is the length of the texture in which the segment is located. For example, the paragraph is a part of a musical work that expresses a complete emotion, and "verse", "chorus", "bridge" and the like refer to paragraphs. The texture refers to a relatively stable structure of several playing methods that exist in a paragraph. In addition to segmenting the segment based on the division granularity of paragraphs and textures, the segment can also be used as a smaller division granularity and further divided into segments.

[0059] In one embodiment, it also includes: by performing the attribute annotation on the music file, first annotation information related to the structure and second annotation information related to the pitch are obtained; wherein the first annotation information includes: annotation information for the paragraph respectively; wherein the attribute annotation for the texture, the segment and the slice is used as part of the annotation information of the paragraph to describe the structural attributes of the paragraph; the second annotation information includes: annotation information obtained by performing attribute annotation on the track and the note respectively. For example, for the first annotation information, for the paragraph, the category of the paragraph needs to be marked, that is, whether it corresponds to the main song or the chorus of the song. There is at least one texture in a paragraph; for the texture, each texture needs to be marked with the maximum length allowed after the texture length changes, the minimum length, the adjustment priority, and whether it can be skipped; for the segment, it is necessary to mark whether the content played by the segment is the main melody or the empty segment, and the instrument used, and the length of the segment is the length of the texture in which it is located; for the slice, the slice is the smallest strong and weak cycle unit in the instrument performance, and it is necessary to mark the placeholder type including any one of the starting slice, the ending slice, the fixed slice and the regular slice. For another example, the second annotation information includes: marking track attributes and note attributes respectively. The track attributes include instrument category, range, etc.; the note attributes include: note pitch-related attributes, such as the chord to which the note belongs, the part to which the note belongs, the note type, etc.

[0060] In one embodiment, the method of reorganizing the structure of each material contained in the arrangement template and re-adjusting the pitch of the notes according to the music information to achieve the arrangement process and obtain the target object includes: reorganizing the structure of each material contained in the arrangement template according to the paragraph and the length of the texture corresponding to the music information to be arranged, obtaining multiple segments matching the paragraph and the length of the texture, and the multiple segments are the structural information constituting the target object; re-adjusting the pitch of the notes of the multiple segments one by one according to the music information to be arranged, and obtaining the target object. For example, the first step is to reorganize the structure (i.e., rearrange the multiple segments), i.e., rearrange the segments according to the paragraph and the length of the texture of the music information to be arranged. Specifically, the paragraph matching can be performed first, such as first matching the paragraph in the matching template with the paragraph of the music information to be arranged; then, the length of the texture can be dynamically planned and allocated, such as allocating the length of each texture according to the length of the music information to be arranged; finally, the multiple segments can be rearranged according to the first annotation information related to the structure (the rearrangement can be achieved according to the performance segment). The second step is to readjust the pitch of the notes of the multiple segments one by one (i.e., adjust the pitch of each segment). Specifically, a pitch search can be performed first. At this time, each segment has already corresponded to a part of the chords of the music information to be arranged, and then all possible new segments are first found by pitch search; then, the new segments obtained can be connected using dynamic rules, so that the target object (new MIDI file, which is already a complete new accompaniment) can be obtained. Furthermore, it can also include: for a performance segment in the music information to be arranged, the accompaniment is changed to a solo as needed.

[0061] Application examples:

[0062] The processing flow of the first embodiment of the present application includes the following contents:

[0063] Figure 2 It is a schematic diagram of the process of automatic arrangement in an application example according to an embodiment of the present application. The process is divided into two parts: data preparation and music generation. Among them, in data preparation, the musician first produces digital music, and then the annotation personnel (who can be the musicians themselves) slice and annotate the musicians' works, add the information required for the generation algorithm, and finally store these materials in the database as an arrangement template. In music generation, it is the user-oriented part. The user selects the required template from the database and inputs the music information to be arranged into the system. Finally, the automatic arrangement system generates a complete digital music file. The process is described in detail below.

[0064] 1. Digital music produced by musicians in data preparation

[0065] In today's digital music era, most musical instruments have alternatives with MIDI output capabilities. For example, pianists can use MIDI keyboards to perform, wind instrument players can use electric blowpipes, drummers can use electronic drums, and even string instruments (guitars, violins) have dedicated peripherals for collecting MIDI signals. With the support of various powerful soft sound sources, using only MIDI can achieve effects comparable to or even better than those of real instrument recordings.

[0066] In this process, the MIDI signals can be recorded through a digital audio workstation (DAW), and can be easily adjusted, replaced, mixed, etc. Compared with recording sounds directly in a concert hall, such digital music has become a major component in modern music production and is developing rapidly.

[0067] Figure 3 is a schematic diagram of a typical MIDI clip in a piano roll according to an application example of an embodiment of the present application. Figure 4 According to an application example of an embodiment of the present application, Figure 3 Schematic diagram of the annotation results presented by the annotation, refer to Figure 3-Figure 4 As shown, Figure 3 Any rectangle in the diagram can represent a note, the left and right sides of the note represent the start and end time of the note, the vertical position of the note represents the pitch, and the strength of each note can also be distinguished by different identifiers (such as different color identifiers). Figure 4 right Figure 3 The chord progression and note types are marked. Among them, the area 11 represents the chord notes, and the area 12 represents the bass notes.

[0068] Specifically, Figure 3 As shown, the above MIDI file, which is typically played by a MIDI keyboard, contains not only the start and end time, pitch and intensity of each note, but also includes information such as: 1) lyrics that change over time; 2) key signatures that change over time; 3) time signatures that change over time; 4) speeds that change over time; 5) names of each track; 6) timbre numbers of each track; 7) control values ​​that change over time in each track; 8) start time, end time, pitch, and intensity of each note in each track. Considering the calculation involved in the performance content, this application can input music projects that only allow pure MIDI production (relative to recording). Apart from this restriction, musicians can freely create in DAW, which is no different from normal music production.

[0069] 2. Data preparation involves segmenting and labeling the digital music produced above

[0070] Figure 5 is a schematic diagram of a multi-track MIDI project in an application example according to an embodiment of the present application, such as Figure 5 As shown in FIG. 1 , the multi-track MIDI is composed of several MIDI tracks, including piano, guitar, bass and drum tracks. Each small rectangle represents a note. From left to right is the time, and the vertical position of the rectangle represents the pitch. Furthermore, in addition to the information required for the user to automatically arrange the music provided by the MIDI project schematic, the project can also be processed manually or by other automated methods. Figure 6 According to an application example of an embodiment of the present application, Figure 5 The schematic diagram of segmentation is as follows: Figure 6 The example shown contains two paragraphs and two textures. Figure 5 The middle area 21 indicates a segment, and the dotted lines 22 indicate the division of slices.

[0071] In this process, an existing MIDI project can be used as the basis for arranging another song. This requires two steps. First, in order to adapt to any length of arrangement tasks, the MIDI project needs to be divided into small pieces and then reorganized according to certain rules. Secondly, the chord progression and melody of each song are different, so the pitch of each note in the reorganized new project needs to be adjusted. The above segmentation and marking are just preparations for these two operations.

[0072] Syncopation and annotation is the process of disassembling existing music projects, while automatic arrangement is the process of reorganizing the disassembled materials. To some extent, these two processes are inverse to each other. However, in order to minimize system coupling, when annotating, you can focus only on the music information of the MIDI project itself, rather than the algorithm information required for automatic arrangement. In music theory, there are already some theories about the horizontal paragraph structure and vertical pitch arrangement of music. Annotation requires the annotator to analyze the MIDI project and complete this music theory information.

[0073] As far as the segmentation of MIDI projects is concerned, in terms of time, a MIDI project can be divided into two levels, namely "paragraphs" and "textures". In addition to paragraphs and textures, each MIDI track is further divided into a smaller granularity called "slices". Among them, a paragraph is a part of a musical work that expresses a complete emotion, such as the "verse", "chorus", "bridge" and so on that we say in daily life. The texture refers to a relatively stable structure of several playing methods in a paragraph. And a slice refers to the smallest strong and weak cycle unit in the performance of an instrument. For example Figure 6 , that is right Figure 5 The result of structural division. Paragraphs are relatively easy to judge in pop music. Texture can also be determined from the thumbnail of the MIDI track. Slices are usually a measure. All three exist objectively in music. The part of a MIDI track within a texture is called a "performance segment", or simply a "segment". The information of the segment can be divided into the performance method in time and the chord progression in pitch. It is the various chord progressions and performance methods that provide a rich listening experience for music.

[0074] As for the annotation of MIDI projects, after dividing the paragraphs, textures and slices of the MIDI project, it is necessary to further annotate the music theory information. The music theory information includes both structural and pitch information. Among them, the structural information refers to the paragraphs, textures, fragments and slices determined in the previous step. The properties that need to be set (the annotation properties related to the paragraphs, textures, fragments and slices) are shown in Table 1.

[0075]

[0076] Table 1

[0077] The paragraph needs to indicate its category, i.e. whether it corresponds to the verse or chorus of the song. There is at least one texture in a paragraph. When migrating to a new song, the paragraph length may change, so the lengths of the textures will also be reallocated. Each texture needs to indicate the maximum and minimum lengths allowed after the change, the priority of adjustment, whether it can be skipped, and other information.

[0078] For each segment, it is necessary to indicate whether the content played by the segment is the main melody or an empty segment. Obviously, these two types of segments do not need to be generated when re-arranging. Since different instruments may have different playing algorithms, the instrument used needs to be indicated. The length of the segment is the length of the texture in which it is located. After the texture is allocated with a good length, the segment needs to fill these lengths. There are two ways to fill them, loop (or tile) and stretch. With the number of groups, you can fill any length. Finally, the segments between a song are not independent of each other, so several common relationships between segments are designed: copy, high / low octave, bind, and merge. Copy and high / low octave are literal meanings. Binding means that the slices of the two segments must correspond one to one when regenerating. Merging means that if the two segments belong to the same track and are adjacent to each other, the two segments are merged into a large segment for generation. If the segments are regarded as points, the segment relationship is the undirected edge between these points. As long as these edges do not produce closed loops, they will not contradict each other.

[0079] Slices are the material used to fill in a piece. Because fills are often added at specific positions during performance, we have designed three special slices, namely, start, end and fixed, which only appear at specific positions. The rest are regular slices.

[0080] In addition to adjusting the structure, in order to make the automatic arrangement of the automatic arrangement system conform to the chords of the new song, the pitch of the notes of each performance segment needs to be adjusted. The automatic arrangement should be able to retain the performance information of each segment of the arrangement template and replace the chord progression information. Therefore, in addition to the structural information, It is also necessary to mark the pitch information of the MIDI project to clarify which chord each note belongs to and its properties. When playing, different instruments often have different playing preferences, so it is necessary to indicate the instruments of each track. In addition, users usually play the bass sound of chords at a fixed position in the slice, and may also play sounds other than chords, so it is also necessary to mark the category of each sound. In the pitch-related annotation attribute example of the present application, the track attributes and the attributes of the notes are shown in Table 2.

[0081] track note Instrument category: keyboard, guitar, etc. The chord range Belonging to the voice part Note type: bass, chord, mode, fixed

[0082] Table 2

[0083] The track needs to indicate the instrument used and the range of the instrument. Each note needs to indicate the chord and part it belongs to, as well as the note type. Common note types include bass notes, chord notes, mode notes, and fixed pitch. Fixed pitch means that the pitch remains unchanged before and after automatic arrangement, for example, the notes of the drum set remain unchanged. If the mode notes are marked, the optional range of this note after the chord is replaced is any within the mode (usually 7 to 8 candidates), and if the chord notes are marked, this note can only choose the notes within the chord after the chord is replaced (usually 3 to 7), and if the bass notes are marked, the optional range is only the bass note of the new chord. Figure 4 The figure below is an example of a fragment notation, which has a chord progression and each note is represented by area 11 for the chord tone and area 12 for the bass tone.

[0084] At this point, the data preparation process is completed. A MIDI music project, structural segmentation, paragraphs, textures, fragments, slices, track properties, chord progressions and the properties of each note can be stored in the database as a whole, as an arrangement template.

[0085] 3. Input the music information to be arranged into the automatic arrangement system in music generation

[0086] In this process, a new piece of music information can be input, that is, the new music information to be arranged can be input. In order to apply the previously marked MIDI project, the music information should include paragraphs, time signatures, speed, modes, chords and melody. Figure 7 is a schematic diagram of music information to be arranged in an application example according to an embodiment of the present application, such as Figure 7 As shown, in an example of the music information to be arranged, it includes paragraph, tempo, time signature, mode, chord and melody information.

[0087] 4. In music generation, the automatic arrangement system generates a complete digital music file

[0088] In this process, you can select an arrangement template and enter music information, and the system can automatically arrange music based on the two. In order to apply the complete arrangement template to a new song, two steps are required. The first step is to rearrange the slices. That is, rearrange the slices according to the paragraphs and their lengths of the new song. Specifically, first use the paragraphs in the matching template to match the paragraphs of the new song, then allocate the length of each texture according to the new length, and then arrange the slices one by one. The second step is to adjust the pitch of each segment. At this point, each segment has corresponded to a part of the chord progression of the new song. First, use the pitch search to find all possible solutions, and then use the dynamic programming algorithm to connect the possible solutions to get new segments. In this way, a complete new accompaniment can be obtained.

[0089] Figure 8 FIG. 1 is a flow chart of automatically arranging music by changing accompaniment to solo in an application example of an embodiment of the present application. Figure 8 As shown in the figure, for a performance segment, the accompaniment can be changed to solo as needed. The following is a detailed description of the operation of each major step.

[0090] As far as paragraph matching is concerned, the song to be arranged, the paragraphs and their order, etc. may be different from the template, so the template is required to cover as many paragraph types as possible so that they can be matched when used.

[0091] As far as dynamic planning of allocating texture length is concerned, after the paragraph length is specified by the new song, if there are multiple textures within the paragraph of the template, the texture length within the paragraph needs to be reallocated. Fig. 9 is a schematic diagram of the distribution of the length of the texture in an application example according to an embodiment of the present application, such as Fig. 9 In the example shown, the adjustment priority of the texture length is equal. According to experience, the texture length of 4 and its multiples will sound more complete. Therefore, when changing the paragraph length, the allocation algorithm can meet the following conditions as much as possible: 1) Each texture should maintain its original length ratio as much as possible; 2) Each texture should be a multiple of 4 as much as possible. For example, if there are two textures with 4+4 bars and the total length is now 10 bars, it is better to allocate them to 4+6 bars than 5+5 bars; 3) Some textures are more universal, so the length of these textures should be changed first when changing the length. To this end, the texture-related attributes in the annotation can be fully utilized to reallocate the length of the texture. A three-level priority is designed. When changing the length, the textures with high priority are adjusted first, and the length of the textures with the next priority will be changed until the length of these textures reaches the limit. If the new paragraph is longer than the original paragraph, only the following situations are possible:

[0092] a. Only adjusting the first-priority paragraphs can meet the requirements, and the length of the second and third-level textures remains unchanged;

[0093] b. All first levels are set to the maximum length, all third level lengths remain unchanged, and the second level is reallocated;

[0094] c. The first and second levels are both of maximum length, and the third level is reallocated.

[0095] d. Unable to meet the requirements.

[0096] The situation where the new paragraph is shorter than the original paragraph is similar. Therefore, if the requirement can be met, it is only necessary to allocate between textures of the same priority, thus satisfying the above condition 3).

[0097] Assuming the priority to be adjusted is p, all textures less than p are set to the limit value, and all textures greater than p maintain their original length. For textures with priority p, dynamic programming algorithm can be used to allocate them. The dynamic programming equation is as follows:

[0098]

[0099] Among them, f(n,l) is the score when only the first n textures are considered and the total length is l bars, w n (k) is the score when the length of the nth texture is k bars. n (k) only needs to meet two conditions: first, the score is higher when it is close to proportional distribution, and second, the score is higher when it is a multiple of 4. The value range of k can be easily obtained from the minimum and maximum lengths in the texture annotation attributes and whether these three values ​​can be skipped.

[0100] As far as arranging slices according to annotations is concerned, that is, rearranging slices fragment by fragment, the length of the texture allocation obtained from the previous step is the length of each track fragment. Next, each fragment needs to use its own slice to fill this length. First, due to the existence of fragment relationships, each fragment may have dependencies (for example, the source of the copy in the case of a copy relationship). Since the fragment relationship is required to be acyclic during annotation, the source of the fragment relationship (root node) can always be found, and these fragments can be generated first. In order to increase the diversity of the arrangement results, the concept of slice grouping is introduced, that is, the number of groups in the annotation. Slices belonging to the same group can be replaced with each other. According to the annotation, the filling methods of the paragraph are looping and stretching, which correspond to the stable fragments and the gradually increasing / decreasing fragments of the music respectively. Depending on the loop and stretch, the definition of the grouping is different. In the loop method, the number of groups refers to the length of the loop. In the stretch method, the number of groups refers to the number of layers of intensity. Fig.10 2 is a schematic diagram of different effects of looping and stretching from 8 slices to 16 slices when the number of groups is 4 according to an application example of an embodiment of the present application, such as Fig.10 In the example shown, the squares represent slices and the numbers represent groups. Slices in the same group can be replaced with each other, showing the different effects of doubling the length when looping and stretching with the same original length and the same number of groups.

[0101] At this point, the structure and rhythm of the automatic arrangement have been completed. Next, you need to adjust the pitch piece by piece.

[0102] As for pitch search, it refers to an algorithm that reasonably migrates to a new chord sequence while retaining the original segment performance method. Through this algorithm, a large number of results of the same performance segment after chord transformation can be quickly obtained. At the same time, these segments not only meet the requirements of the new chord sequence, but also retain the texture of the original segment and conform to the performance method of a specific instrument. The following is a detailed description of the pitch search algorithm process.

[0103] 1. Annotation constraints

[0104] First, after the new music information is input based on the third step, the new chord to which each note belongs is divided by time, and then the optional range of each note can be obtained by using the annotation information obtained in the second step, which is recorded as M. p , where p traverses every note in the performance segment, that is, the pitch range of each note under the new chord sequence. And let M = {M p |p=0,...,N p -1}, where N p is the total number of notes in the performance segment.

[0105] 2. Constraints of musical instruments

[0106] The performance fragment is obtained by playing a musical instrument. Because the performance is inevitably constrained by the posture and movement of the human body, the structure of the instrument determines the stacking method of the notes that can be played by the instrument. For example, it is easy to play five notes CDEFG at the same time on a keyboard instrument, but it is almost impossible on a guitar. In order to describe this limitation, a set of constraint systems needs to be established.

[0107] The general form of the constraint is as follows. Given the total number of tones N p When , the constraint G is:

[0108]

[0109] Among them, H p,n is a set of integers, indicating that in the nth set of constraint solutions G n Next, the optional range of the pth tone. G is the total number of constraint solutions. The following is a specific form of the above constraint, which is the hand shape when playing a musical instrument.

[0110] Fig.11 FIG. 12 is a schematic diagram of constraint parameters of a guitar in an application example according to an embodiment of the present application, and the numbers represent semitone intervals. Figure 11-Figure 12 , we can let the positive integer N h Indicates the number of "hands" required to play the instrument. Positive integer Nf,h is the number of "fingers" allowed for the h-th hand, that is, the maximum number of notes allowed to sound at the same time. h =(I 0,h ,I 1,h ,I 2,h ,...,I Nf,h-1,h ) is the "activity center" of each "finger" of each hand, that is, the pitch aligned with the lowest note of this hand. min , R max To constrain the movable pitch range of each finger to [R min ,R max For common musical instruments, appropriate parameters can be selected to describe them. For example, for keyboard instruments, N h =2,N f,0 =N f,1 =5, by comparing the finger width with the width of the normal keyboard, you can set I 0 =p 0 +(0,2,4,6,8),I 1 =p 1 +(0, 2,4,6,8), R min =-3, R max =3, where p 0 and p 1 These are the pitches corresponding to the base of the left pinky and right thumb. For guitar, you can set N h =1,N f,0 =6, I=p+(0,5, 10,15,19,24), R min =0, R max =3, p is the pitch of the index finger on the sixth string. Figure 11-Figure 12 The sources of various parameters on the keyboard and guitar are shown respectively. For a string quartet, although a single guitar is basically just a single instrument and cannot form harmony, a string quartet can be seen as four "hands" playing on the keyboard, so appropriate parameters can also be selected for modeling.

[0111] In actual performance, not all fingers are placed on the instrument all the time. For a specific number of notes, there are often some fixed hand shapes. For example, when only 5 notes are played, the piano often uses the little finger and index finger of the left hand and the thumb, middle finger and little finger of the right hand to complete the performance. Therefore, a subset of I can be used to represent a hand shape. The hand shape can be enumerated by permutations and combinations when the total number of notes is known, or it can be preset in advance.

[0112] 3. Conversion method

[0113] Fig.13is a schematic diagram of chord conversion of a piano performance segment in an application example according to an embodiment of the present application, such as Fig.13 As shown, for a piano performance segment, {m i} is the original chord sequence in this fragment, {n j} is the new chord sequence to be migrated. This is a many-to-many conversion problem, and the two sets of time points can be taken as the union set {t k}, only focusing on one segment, thus simplifying the problem to a one-to-one conversion problem. As long as this one-to-one problem can be solved, the original many-to-many problem can be solved by stringing them together one by one. In addition, two assumptions are made. 1) Before the conversion, if two segments belong to the same t k The pitch of the two sounds is the same, so they should remain the same after conversion. k If the pitch of one of the notes is higher than the pitch of the other note, the pitch of the first note after conversion cannot be lower than the pitch of the second note after conversion. Let p represent the pitch before conversion and q represent the pitch after conversion. The above condition can be recorded as C:

[0114]

[0115] Under this assumption, the original problem becomes a search problem, which can be solved by depth-first search. The pseudo code is as follows:

[0116]

[0117] Among them, the depth-first search function search is defined:

[0118]

[0119] Among them, N p,k Yes k The total number of notes in . After searching, we can get the k A series of playing methods, which meet the requirements of chord migration and the constraints of hand shape. Fig.14 is a schematic diagram of search results for a Cm9 chord in a six-note column chord according to an application example of an embodiment of the present application, such as Fig.14 As shown, some of the search results for Cm9 in a six-note column chord are basically ergonomic.

[0120] As for the use of dynamic rules to connect new segments, that is, dynamic planning splicing, through the pitch search in the previous step, each t kSeveral search results can be found as alternatives. An expert system is formed through music theory analysis, simulated performance and other scoring, and then a dynamic programming algorithm is used to select and connect the segments with good effects, in line with the public aesthetics and smooth connection, and then the performance segments under the new chord sequence can be obtained.

[0121] Fig.15 is a schematic diagram of selecting the best path in a multi-stage dynamic planning stage according to an application example of an embodiment of the present application, such as Fig.15 As shown, t k The search result is recorded as c l,k , each c l,k They are all a length of N p,k A vector representing t k The final generated performance segment is actually an optimal path found in the multi-stage dynamic planning stage, where the dotted line is a possible path and the solid line represents the optimal path.

[0122] Because each t k There may be a large number of results, which are not suitable for direct search, so you can ignore the cross t k The connection quality of k The quality of itself, and the adjacent t k The connection quality between them is divided into two parts. In this way, the optimal path problem will satisfy the optimal substructure property and can be solved by dynamic programming. Let f(l,k) represent the optimal path problem when only t is considered. 0 ,t 1 ,...,t k-1 ,t k , and ends in c l,k The optimal connection score, then the dynamic programming equation can be obtained as follows:

[0123]

[0124] Among them, w(c l’,k-1 ,c l,k ) is a function to evaluate the quality of the connection between any two search results, and v(c l,k ) is a function that evaluates the quality of a search result. In this way, the final optimal path can be obtained through max(f(l,N k -1)). The form of w and v can be flexible and can also contain parameters required for generation. For example, for function v, the following requirements can also be met:

[0125] a. You cannot conflict with the main theme. If you do, you will be deducted points.

[0126] b. Meet the low-range limit of the instrument's timbre. If not, points will be deducted.

[0127] c. When all the notes are played from front to back, the notes within the chord should appear first as much as possible to make the properties of the chord clear.

[0128] d. Add or subtract points according to the color preference parameters specified during generation. For example, when the requirement is a simple and bright accompaniment, more chord tones can add points, but when the requirement is a complex harmony such as jazz, more chord tones can add points.

[0129] For the connection quality evaluation function w, the following requirements can also be met:

[0130] a. Depending on whether the generated parameters are stable or not, add or subtract points for the degree of change in the connecting range.

[0131] b. Consider the drastic change in hand shape before and after, usually the smaller the better.

[0132] c. For the outer parts of polyphonic instruments, the smoother the connection, the better, and it is better for the upper and lower outer parts to be in opposite directions than in the same direction.

[0133] The two functions v and w can be further refined to form an expert system, so that the optimal connection in the sense of these two functions can be obtained.

[0134] As for changing the accompaniment to a solo as needed, it is an optional part, considering that sometimes there is a need to generate an instrumental solo. In this case, further post-processing is required to convert the polyphonic instrument as the main melody accompaniment in the previous step into a polyphonic instrument solo. Fig.16 is a schematic diagram of an accompaniment in an application example according to an embodiment of the present application, Fig.17 is a schematic diagram of a solo in an application example according to an embodiment of the present application, see Figure 16-Figure 17 After the above pitch adjustment steps, we can get an example of an accompaniment segment, which has the same pitch distribution as the main melody. Fig.16 Even if they are played with the same MIDI instrument, it still sounds like two instruments are playing. To convert this accompaniment and main melody into Fig.17 The solo of the polyphonic instrument shown mainly starts from the two aspects of rhythm and range.

[0135] In most cases, in order to highlight the auditory sense of the melody, the main melody will be arranged in the treble part. Therefore, the conversion method proposed in this application only considers the situation where the "hand" with the highest pitch is responsible for playing the treble part. After a polyphonic instrument obtains the accompaniment through the above steps, only the notes responsible for the "hand" responsible for the highest pitch part are adjusted to obtain a solo. When people play solos, they are often affected by the main melody and change the rhythm of the accompaniment notes so that their rhythm is consistent with the main melody. Secondly, since one hand is required to play the main melody and chords at the same time, their range of sound will also be limited by the hand shape. Through these two adjustments, you can get an effect that sounds more like a solo. The following is a detailed description of the conversion process of changing the accompaniment to a solo.

[0136] First, find out the notes played by the original highest-pitched "hand" (such as the right hand of the piano) and paste the main melody on it, such as Fig.16 Then, for the melody on the re-beat, if there are accompaniment notes within the 16th notes before and after it, these accompaniment notes are adjusted to the vicinity of the melody. Finally, the algorithm for rearranging the fingerings using the hand shape is used. This conversion process is similar to the pitch search and dynamic programming connection process mentioned above, except that: since the main melody needs to be played at the same time, it is possible to play the accompaniment notes at the same t k In the range, using only one hand shape can no longer meet the requirements, and the hand shape needs to change with the main melody. However, in the actual performance, not every main melody note needs to change the hand shape, but the hand shape should not be changed as much as possible. Therefore, it is necessary to determine which ranges only use one hand shape, that is, for t k To further subdivide, Fig.18 is a schematic diagram of further dividing tk according to hand shape in an application example of an embodiment of the present application, such as Fig.18 As shown, in t k In the process of further division according to hand shape, once the scope of subdivision can be determined, the arrangement of these accompaniment notes can be re-planned completely according to the steps of pitch search plus dynamic planning connection.

[0137] The following is a greedy search strategy for determining the coverage of a single hand shape. The basic idea of ​​the strategy is to find t k The highest note of the corresponding main melody, then starting from this note, based on the highest note in the hand shape to play this melody note, find the maximum range that this hand shape can cover, and then if there are remaining melody notes before and after, continue this process for the remaining part. The pseudo code is as follows:

[0138]

[0139] Define a recursive function get_range to find the range:

[0140]

[0141] Through the above recursive algorithm, we can find a hand shape that can be kept as still as possible, and use this hand shape (i.e. Gm) to re-do the pitch search process. After dynamic planning, we can turn the accompaniment into a solo. At this time, due to the limitation of the main melody, the w and v functions of the dynamic planning can be appropriately adjusted.

[0142] At this point, the automatic arrangement is complete, and the MIDI file can be exported for DAW playback and audio file generation.

[0143] The above processing flows of this application example have the following beneficial effects:

[0144] This application designs an automatic arrangement process and algorithm based on existing MIDI engineering. Through the method provided by this application, an existing arrangement can be easily applied to a piece of music to be arranged. The conversion process is efficient and supports solo mode. Users do not need to understand the knowledge related to arrangement and instrumentation, which greatly improves the production efficiency of music. In addition, it also has the following advantages:

[0145] 1. You can use different parameters to achieve the performance effects of different instruments;

[0146] 2. Dynamic programming can achieve the optimal effect under the generated parameters, with high originality;

[0147] 3. The style, paragraphs, complexity, etc. of the generated results are highly controllable;

[0148] 4. It can be converted into a solo instrument, and the conversion result conforms to human fingering;

[0149] With the support of this automatic arrangement system, arrangement becomes efficient and reasonable. It can be applied in many fields. First, the use of this application can greatly reduce the difficulty of music production, allowing musicians to focus their energy on where musical inspiration is really needed. After quickly producing a music sample, it can also further inspire the inspiration of musicians. Some music styles, such as rap, are accompaniment first and then lyrics. Secondly, using this application, you can quickly produce an accompaniment that meets the needs of such musicians. And a marked template can be used repeatedly, which can greatly improve the efficiency of music production.

[0150] In terms of video soundtracks, existing videos all use existing music for soundtracks. However, in addition to possible copyright issues, the use of existing music may not be able to match the length, atmosphere and rhythm of the video well. However, it is obviously unrealistic to require all video producers to master music production and compose music for their own videos. Using this automatic arrangement system, you can choose the style, paragraphs and other time points by yourself, so as to easily produce soundtracks that match the video. Since the MIDI file finally output by this application is ergonomic, it can actually be played. Therefore, this application can adapt existing music into instrumental solos, which can be used in instrument teaching. Among them, the process of segmentation and annotation is the process of disassembling a music project. This process allows the annotation personnel to fully learn the writing techniques of musicians, and it is also a good arrangement learning tool.

[0151] Since the automatic arrangement of this application is generated quickly and all parameters are controllable, based on this application, combined with deep learning music writing, a real automatic performance machine can be realized. Sensors can also be used to sense the environment and generate music that conforms to the surrounding environment in real time.

[0152] According to an embodiment of the present application, an information processing device is provided. Fig.19 is a schematic diagram of the structure of an information processing device according to an embodiment of the present application. Fig.19 As shown, it includes: a template acquisition module 51, used to obtain an arrangement template based on a music file, wherein the music file includes a file edited based on a musical instrument MIDI; an information acquisition module 52, used to obtain music information to be arranged; and a synthesis processing module 53, used to obtain a target object based on the music information and the arrangement template.

[0153] In one embodiment, it further includes a segmentation and annotation module for segmenting and annotating the music file to obtain the arrangement template; wherein the arrangement template includes: paragraphs, textures, fragments, slices, tracks, and notes.

[0154] In one embodiment, the synthesis processing module is used to restructure and readjust the pitches of notes of various materials included in the arrangement template according to the music information, so as to realize arrangement processing and obtain the target object.

[0155] In one embodiment, the segmentation and annotation module is also used to obtain the paragraphs by performing the segmentation process on the music file; each of the paragraphs contains at least one texture; the length of the segment in each of the paragraphs is the length of the texture in which the segment is located.

[0156] In one embodiment, the segmentation and annotation module is also used to obtain first annotation information related to the structure and second annotation information related to the pitch by annotating the music file with the attributes; wherein the first annotation information includes: annotation information for the paragraphs respectively; wherein the attribute annotations performed on the texture, the fragment, and the slice are used as part of the annotation information of the paragraph to describe the structural attributes of the paragraph; and the second annotation information includes: annotation information obtained by annotating the tracks and the notes with the attributes respectively.

[0157] In one embodiment, the synthesis processing module is used to structurally reorganize the various materials contained in the arrangement template according to the paragraphs and texture lengths corresponding to the music information to be arranged, so as to obtain multiple segments matching the paragraphs and texture lengths, wherein the multiple segments are structural information constituting the target object; and according to the music information to be arranged, the pitches of the notes of the multiple segments are readjusted one by one to obtain the target object.

[0158] The functions of each module in each device in the embodiment of the present application can be found in the corresponding description in the above method, and will not be repeated here.

[0159] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0160] like Fig. 20 , is a block diagram of an electronic device for implementing the information processing method of an embodiment of the present application. The electronic device can be the aforementioned deployment device or proxy device. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples, and are not intended to limit the implementation of the present application described and / or required herein.

[0161] like Fig. 20As shown, the electronic device includes: one or more processors 801, memory 802, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are interconnected using different buses and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig. 20 A processor 801 is taken as an example.

[0162] The memory 802 is a non-transient computer-readable storage medium provided in the present application. The memory stores instructions executable by at least one processor to enable the at least one processor to perform the information processing method provided in the present application. The non-transient computer-readable storage medium of the present application stores computer instructions, which are used to enable a computer to perform the information processing method provided in the present application.

[0163] The memory 802 is a non-transient computer-readable storage medium that can be used to store non-transient software programs, non-transient computer executable programs and modules, such as program instructions / modules corresponding to the information processing method in the embodiment of the present application. The processor 801 executes various functional applications and data processing of the server by running the non-transient software programs, instructions and modules stored in the memory 802, that is, implementing the information processing method in the above method embodiment.

[0164] The memory 802 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 802 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 802 may optionally include a memory remotely arranged relative to the processor 801, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0165] The electronic device of the information processing method may further include: an input device 803 and an output device 804. The processor 801, the memory 802, the input device 803 and the output device 804 may be connected via a bus or other means. Fig. 20 The example of connecting through bus is taken in the following.

[0166] The input device 803 can receive input digital or character information, and generate key signal input related to user settings and function control of the electronic device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator rod, one or more mouse buttons, a trackball, a joystick and other input devices. The output device 804 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display and a plasma display. In some embodiments, the display device may be a touch screen.

[0167] Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0168] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for programmable processors and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or means (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0169] To provide interaction with a user, the systems and techniques described herein may be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0170] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communications network). Examples of communications networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0171] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0172] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document does not limit this.

[0173] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. An information processing method, characterized in that: The method comprises: Acquire a music arrangement template based on a music file, wherein the music file includes a file edited based on a musical instrument digital interface (MIDI); Get the music information to be arranged; Obtaining a target object according to the music information and the arrangement template; The step of obtaining the target object according to the music information and the arrangement template includes structurally reorganizing the various materials contained in the arrangement template according to the paragraphs and the texture lengths corresponding to the music information to be arranged, and obtaining a plurality of segments matching the paragraphs and the texture lengths, wherein the plurality of segments are the structural information constituting the target object; structurally reorganizing the various materials contained in the arrangement template and readjusting the pitches of the notes according to the music information, so as to realize the arrangement process and obtain the target object; Among them, the structural reorganization of each material contained in the arrangement template includes allocating texture length through a dynamic programming algorithm; the readjustment of note pitches includes readjusting the note pitches of the multiple segments one by one through a pitch search algorithm.

2. The method according to claim 1, characterized in that Also includes: The music file is segmented and attributed to obtain the arrangement template; wherein, The arrangement template includes: paragraphs, textures, fragments, slices, tracks, and notes.

3. The method according to claim 2, characterized in that Also includes: By performing the segmentation process on the music file, the paragraphs are obtained; Each of the paragraphs contains at least one of the textures; The length of the segment in each of the paragraphs is the length of the texture in which the segment is located.

4. The method according to claim 2, characterized in that: Also includes: By performing the attribute annotation on the music file, first annotation information related to the structure and second annotation information related to the pitch are obtained; wherein, The first annotation information includes: annotation information for each of the paragraphs; wherein the attribute annotations for the texture, the segment, and the slice are used as part of the annotation information for the paragraph to describe the structural attributes of the paragraph; The second marking information includes: marking information obtained by marking attributes of the track and the note respectively.

5. An information processing device, characterized in that: The device comprises: A template acquisition module, used to acquire an arrangement template based on a music file, wherein the music file includes a file edited based on a musical instrument digital interface (MIDI); An information acquisition module is used to acquire music information to be arranged; A synthesis processing module, used for obtaining a target object according to the music information and the arrangement template; The step of obtaining the target object according to the music information and the arrangement template includes structurally reorganizing the various materials contained in the arrangement template according to the paragraphs and the texture lengths corresponding to the music information to be arranged, and obtaining a plurality of segments matching the paragraphs and the texture lengths, wherein the plurality of segments are the structural information constituting the target object; structurally reorganizing the various materials contained in the arrangement template and readjusting the pitches of the notes according to the music information, so as to realize the arrangement process and obtain the target object; Among them, the structural reorganization of each material contained in the arrangement template includes allocating texture length through a dynamic programming algorithm; the readjustment of note pitches includes readjusting the note pitches of the multiple segments one by one through a pitch search algorithm.

6. The device according to claim 5, characterized in that It also includes segmentation and annotation modules for: The music file is segmented and attributed to obtain the arrangement template; wherein, The arrangement template includes: paragraphs, textures, fragments, slices, tracks, and notes.

7. The device according to claim 6, characterized in that The segmentation and labeling module is also used for: By performing the segmentation process on the music file, the paragraphs are obtained; Each of the paragraphs contains at least one of the textures; The length of the segment in each of the paragraphs is the length of the texture in which the segment is located.

8. The device according to claim 6, characterized in that The segmentation and labeling module is also used for: By performing the attribute annotation on the music file, first annotation information related to the structure and second annotation information related to the pitch are obtained; wherein, The first annotation information includes: annotation information for each of the paragraphs; wherein the attribute annotations for the texture, the segment, and the slice are used as part of the annotation information for the paragraph to describe the structural attributes of the paragraph; The second marking information includes: marking information obtained by marking attributes of the track and the note respectively.

9. An electronic device, comprising: at least one processor; as well as A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor so that the at least one processor can execute the method of any one of claims 1-4.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 4.