A method for aligning and merging polyphonic audio

Abstract

The present invention discloses a method, a system and a computer storage medium for aligning and merging multi-voice audio, including: obtaining the performance audio of different voices in a musical score, detecting and extracting the piano sound segments included in each performance audio; extracting the score of each voice, and calculating the reference speed v of the musical score according to the performance duration of each voice audio and the number of beats included in the score mean ; aligning the audio of each voice with the musical score at the reference speed to obtain a first alignment path, performing variable speed adjustment according to the first alignment path, and merging the audio after variable speed adjustment to obtain a first audio; calculating a second alignment path, and performing non-linear speed adjustment on the first audio to obtain a second audio; performing dynamics adjustment on the second audio to obtain a merged audio; adding reverberation to obtain a final audio. The present invention can, while aligning each voice, maximize the retention of the original performance style of each voice, making the merged audio more artistically expressive.

Description

Technical Field

[0001] The present invention relates to the field of audio technology, and more particularly, to a method, system and computer-readable storage medium for aligning and merging multi-voice audio. Background Art

[0002] In recent years, with the popularization of music education and the rapid development of Internet technology, music lovers are no longer limited to practicing or performing alone. Recently, more and more people have begun to seek to play different voices of the same piece of music with different people, such as string quartets, piano-violin duets, or even symphonies, etc., and synthesize them and share them on social platforms.

[0003] However, the performers of each voice have different understandings and processing methods of music works, and the roles played by each voice of the same work are also different. Therefore, the performance audio of each voice will show different speeds, intensities, emotional changes, etc. How to align these different voices according to a certain speed curve, then merge them together, and try to ensure the original performance styles of each voice is not an easy task. There is a high probability that each voice will have long rests, and when the user solos this voice, they usually just pause slightly and then continue to play the passage after the long rest, which brings trouble to the alignment and merging of the audio.

[0004] In the prior art, the Chinese invention patent with the publication number CN107591149A discloses an audio synthesis method, device and storage medium, belonging to the field of multimedia technology. The method includes: obtaining a target original singing audio file corresponding to the accompaniment audio file according to the song information corresponding to the accompaniment audio file to be synthesized; if the first duration difference between the accompaniment audio duration of the accompaniment audio file and the original singing audio duration of the target original singing audio file is less than or equal to a preset threshold, determining the accompaniment-original singing alignment position through the short-time energy algorithm, and if the first duration difference between the accompaniment audio duration of the accompaniment audio file and the original singing audio duration of the target original singing audio file is greater than the preset threshold, determining the accompaniment-original singing alignment position through the short-time cross-correlation algorithm; based on the accompaniment-original singing alignment position, merging the accompaniment audio file and the target original singing audio file into a complete audio file according to the specified audio file format. This prior art is aimed at the merging of accompaniment and singing audio, and does not solve the problem of merging the performance audio of different voices. Summary of the Invention

[0005] In order to overcome the above-mentioned defects of the prior art, the present invention provides a method, system and computer-readable storage medium for aligning and merging multi-voice audio, which can neatly align each voice and retain the original performance style of each voice to the greatest extent, making the merged audio more artistically expressive.

[0006] The primary objective of the present invention is to solve the above technical problems, and the technical solution of the present invention is as follows:

[0007] The first aspect of the present invention provides a method for aligning and merging multi-voice audio, including the following steps:

[0008] Obtain the performance audio of different voices in the music score, and detect and extract the piano sound passages included in each said performance audio;

[0009] Divide the music score according to voices and extract the separate scores of each voice. Calculate the reference speed v of the music score based on the performance duration of each voice audio and the number of beats included in the separate score. mean ;

[0010] Align the audio of each voice with the music score at the reference speed to obtain a first alignment path. Perform speed change adjustment using a phase vocoder according to the first alignment path, and merge the speed-adjusted audio to obtain a first audio;

[0011] Calculate the second alignment path between the reference voice and the music score, and perform non-linear speed adjustment on the first audio according to the second alignment path to obtain a second audio;

[0012] Introduce the dynamics of the reference voice into the second audio using the RMS energy value to obtain a merged audio;

[0013] Add reverberation to the merged audio through room impulse response convolution to obtain the final audio.

[0014] In this solution, the specific process of dividing the music score according to voices, extracting the separate scores of each voice, and calculating the reference speed of the music score based on the performance duration of each voice audio and the number of beats included in the separate score is as follows:

[0015] Divide the music score according to voices to obtain the separate score s of each voice n , analyze the duration of each segment of audio of each voice, that is, the duration of the piano sound passage, parse out the number of beats included in the music score corresponding to each piano sound passage, and calculate the performance speed v of the current voice according to the duration of the piano sound passage and the number of beats included. n , take the average value v of the performance speeds of all voices mean , obtain the average performance speed as the reference speed of the music score, that is: n represents the number of voices.

[0016] In this solution, the specific process of aligning the audio of each voice with the music score at the reference speed to obtain a first alignment path, performing speed change adjustment using a phase vocoder according to the first alignment path, and merging the speed-adjusted audio to obtain a first audio is as follows:

[0017] Frame the audio of each voice and align it with the separate score s of the corresponding voice nAlign to obtain the first alignment path d at the frame level n For each voice audio, according to the first alignment path d n Use a phase vocoder to change the speed to obtain a spectrogram s with a reference speed of v mean Consistent audio x n , Add the audio of the n voices after speed adjustment and divide by n to obtain the first audio. n In this solution, when the spectrogram of each voice contains rests, first cut the audio of the voice and the spectrogram of the voice to find the section of the audio containing the piano sound

[0018] , Calculate the total duration t of the rest by combining the number of beats of the rest in the music score with the reference speed v of the music score , And fill in the mute signal of the corresponding duration on the audio between mean to obtain the audio b of this voice with the rest duration filled in , Then align b n with the music score s n to obtain the first alignment path d n , Then use a phase vocoder to change the speed of each voice audio according to the first path d n to obtain audio x consistent with the music score s with a reference speed of v n mean n n n Consistent audio x n Finally, add the adjusted audio of the n voices and divide by n to obtain the merged first audio.

[0019] In this solution, calculate the second alignment path between the reference voice and the music score, and non-linearly adjust the speed of the first audio according to the second alignment path to obtain the second audio. The specific process is as follows:

[0020] Select the audio of the reference voice, and align the audio of the reference voice with the music score with a reference speed of v mean to obtain the second alignment path;

[0021] After non-linearly changing the speed of the first audio with a phase vocoder according to the second alignment path, the new second audio with the playing speed of the reference voice can be obtained.

[0022] In this solution, use the RMS energy value to introduce the dynamics of the reference voice into the second audio to obtain the merged audio. The specific process is as follows:

[0023] Frame the audio signal of the reference voice according to the preset sampling rate, and calculate the RMS energy value of the reference voice audio frame by frame

[0024] Calculate the energy value of each voice audio of the second audio, and scale the energy value of each frame of each voice audio of the second audio to be consistent with the RMS energy value of each frame of the reference voice audio;

[0025] Sum the audio of each voice part of the scaled second audio to obtain a combined audio.

[0026] In this solution, the RMS energy of the i-th frame is expressed as:

[0027]

[0028] N represents the number of sampling points per frame, and x i represents the amplitude value of each sampling point of the signal.

[0029] In this solution, add reverberation to the combined audio through room impulse response convolution to obtain the final audio. The specific process is as follows:

[0030] Select a room impulse response that meets the preset conditions, and convolve the room impulse response with the combined audio to obtain the final audio D final

[0031] D final = Dvv * RIR

[0032] Dvv represents the combined audio, and RIR represents the room impulse response.

[0033] The second aspect of the present invention provides a system for aligning and combining multi-voicepart audio. The system includes: a memory and a processor. The memory includes a method program for aligning and combining multi-voicepart audio. When the method program for aligning and combining multi-voicepart audio is executed by the processor, the following steps are implemented:

[0034] Obtain the performance audio of different voice parts in the musical score, and detect and extract the piano passages included in each performance audio;

[0035] Divide the musical score by voice part and extract the score of each voice part. Calculate the reference speed v of the musical score according to the performance duration of each voice part audio and the number of beats included in the score mean ;

[0036] Align the audio of each voice part with the musical score at the reference speed to obtain a first alignment path. Perform speed change adjustment on the speed-changed adjusted audio using a phase vocoder, and combine the speed-changed adjusted audio to obtain a first audio;

[0037] Calculate the second alignment path between the reference voice part and the musical score, and perform non-linear speed adjustment on the first audio according to the second alignment path to obtain a second audio;

[0038] Introduce the dynamics of the reference voice part into the second audio using the RMS energy value to obtain a combined audio;

[0039] Add reverberation to the combined audio through room impulse response convolution to obtain the final audio.

[0040] In the third aspect of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium includes a method program for aligning and merging multi-voice audio. When the method program for aligning and merging multi-voice audio is executed by a processor, the steps of the method for aligning and merging multi-voice audio are implemented.

[0041] Compared with the prior art, the beneficial effects of the technical solution of the present invention are as follows:

[0042] By preprocessing the performance audio and musical score, the present invention aligns the audio of each voice using the reference speed, then performs audio speed change on the alignment path of the reference voice pair and the musical score, and finally polishes the dynamics and reverberation of the audio to obtain the final audio. The present invention can neatly align each voice and retain the original performance style of each voice to the greatest extent, making the merged audio more artistically expressive. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a flowchart of a method for aligning and merging multi-voice audio according to an embodiment of the present invention.

[0044] Figure 2 It is a flowchart of aligning and merging the first audio using the performance speed of the reference voice according to an embodiment of the present invention.

[0045] Figure 3 It is a flowchart of adjusting the dynamics of the second audio according to an embodiment of the present invention.

[0046] Figure 4 It is a system block diagram of a method for aligning and merging multi-voice audio according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0047] In order to more clearly understand the above objects, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, without conflict, the embodiments of the present application and the features in the embodiments can be combined with each other.

[0048] In the following description, many specific details are set forth in order to fully understand the present invention. However, the present invention can also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited by the specific embodiments disclosed below.

[0049] Embodiment 1

[0050] As Figure 1 shown, the first aspect of the present invention provides a method for aligning and merging multi-voice audio, including the following steps:

[0051] S1 Obtain the performance audio of different voices in the music score, and detect and extract the piano passages included in each said performance audio;

[0052] It should be noted that in the present invention, first obtain the performance audio of different voices in a music score, for example, the performance audio a of n voices n , where each voice is performed by the performers at different speeds and intensities. Then, the piano detection algorithm can be used to detect and extract the piano passages included in each said performance audio a n included

[0053] S2 Divide the music score according to the voices and extract the score of each voice. Calculate the reference speed v of the music score according to the performance duration of each voice audio and the number of beats included in the score mean ;

[0054] It should be noted that the processing of the music score in the present invention is: divide the music score according to the voices and extract the score s of each voice n , analyze the duration of each segment of audio of each voice, that is, the duration of the piano passage, parse out the number of beats included in the music score corresponding to each piano passage, and calculate the performance speed v of the current voice according to the duration of the piano passage and the number of beats included n , take the average value v of the performance speeds of all voices mean , and obtain the average performance speed as the reference speed of the music score, that is: n represents the number of voices.

[0055] It should be noted that the reason for taking the average speed as the reference speed of the music score is: usually, for the case where the difference between the performance speed and the music score speed is too large in the method of aligning audio and music score, the alignment effect will be very poor, which will in turn affect the effect after merging multi-voice audio.

[0056] According to the reference music score speed v mean Fill in the corresponding silent segments in the audio of each voice, so as to obtain the audio of each voice with the silent duration matching the rest in the music score.

[0057] S3 Align the audio of each voice with the music score at the said reference speed to obtain the first alignment path. Use the phase vocoder to perform speed adjustment according to the first alignment path, and merge the speed-adjusted audios to obtain the first audio;

[0058] It should be noted that when the scores of each voice do not contain rests, the score following algorithm can be used to align each frame of the audio of each voice with the score s of the corresponding voice n to obtain the first alignment path d at the frame level n , and perform speed change on the audio of each voice according to the first alignment path d n using the phase vocoder to obtain the same as at vmean The spectral score s of the reference speed n Consistent audio x n After adding the audio of n voices adjusted for speed and dividing by n, the first audio is obtained.

[0059] When the spectral score of each voice contains rests, first cut the audio of the voice and the spectral score of the voice to find the passage where the audio contains the sound of the piano. Combined with the reference speed v of the musical score through the number of beats of the rests in the musical score mean Calculate the total duration t of the rests and fill in the mute signal of the corresponding duration on The audio of this voice between them, and then the audio b of this voice with the duration of the rests filled is obtained. n Then use the musical score following algorithm, such as: dynamic time warping (DTW), musical score following based on particle filtering, etc. to align b n With the musical score s n To obtain the first alignment path d n Then, the audio of each voice is speed-changed according to the first path d n Using a phase vocoder to obtain an audio x that is consistent with the musical score s with v mean As the reference speed n Consistent audio x n Finally, add the adjusted audio of n voices and divide by n to obtain the merged first audio.

[0060] It should be noted that the post-processing of the merged first audio includes: the post-processing includes the polishing of the playing speed, dynamics, and reverb.

[0061] S4 Calculate the second alignment path between the reference voice and the musical score, and non-linearly adjust the speed of the first audio according to the second alignment path to obtain the second audio;

[0062] It should be noted that since the merged first audio of multiple voices is played mechanically according to the musical score speed and lacks musical expressiveness, it is necessary to polish the performance of the first audio with a certain voice as a reference to make the merged audio have the playing speed of the reference voice. At the same time, other classic performance audio of this piece can also be used to polish the first audio.

[0063] Such as Figure 2 The flowchart of aligning and merging the first audio using the playing speed of the reference voice is shown. The specific steps are as follows:

[0064] S401 Select the audio of the reference voice, and align the audio of the reference voice with the musical score with v mean As the reference speed to obtain the second alignment path;

[0065] After non-linearly varying the speed of the first audio using a phase vocoder according to the second alignment path, a new second audio with the playing speed of the reference voice part can be obtained.

[0066] S5 Introduce the dynamics of the reference voice part into the second audio using the RMS energy value to obtain a merged audio;

[0067] After obtaining the second audio, next, introduce the dynamic changes of the reference voice part. The currently merged and speed-changed audio D ref Lacks unified dynamic changes, and each voice part still plays according to its original dynamics. In order to achieve a better musical effect, it is necessary to control the overall dynamics of the merged audio.

[0068] Figure 3 The flowchart for adjusting the dynamics of the second audio is as follows. The specific steps are:

[0069] S501 Frame the audio signal of the reference voice part at a preset sampling rate, and calculate the RMS energy value of the reference voice part audio frame by frame; for example, the sampling rate can be 44100HZ, each frame has N sampling points, and the frame jump is 441 sampling points. The RMS energy of the i-th frame Is expressed as:

[0070]

[0071] N represents the number of sampling points per frame, and x i Represents the amplitude value of each sampling point of the signal.

[0072] S502 Calculate the energy value of each voice part audio of the second audio, and scale the energy value of each frame of each voice part audio of the second audio to be consistent with the RMS energy value of each frame of the reference voice part audio;

[0073] S503 Add the scaled audio of each voice part of the second audio to obtain a merged audio.

[0074] S6 Add reverberation to the merged audio through room impulse response convolution to obtain the final audio.

[0075] The specific process is:

[0076] Select a room impulse response that meets the preset conditions, and convolve the room impulse response with the merged audio to obtain the final audio D final

[0077] D final = Dvv * RIR

[0078] Dvv represents the merged audio, and RIR represents the room impulse response.

[0079] Such as Figure 4As shown, the second aspect of the present invention provides a system for aligning and merging multi-voice audio. The system includes: a memory and a processor. The memory includes a method program for aligning and merging multi-voice audio. When the method program for aligning and merging multi-voice audio is executed by the processor, the following steps are implemented:

[0080] Obtain the performance audio of different voices in the music score, and detect and extract the piano passages included in each performance audio;

[0081] Divide the music score by voice and extract the score of each voice. Calculate the reference speed v of the music score according to the performance duration of each voice audio and the number of beats included in the score mean ;

[0082] Align the audio of each voice with the music score at the reference speed to obtain a first alignment path. Use a phase vocoder to perform speed adjustment according to the first alignment path, and merge the speed-adjusted audio to obtain a first audio;

[0083] Calculate the second alignment path between the reference voice and the music score, and perform non-linear speed adjustment on the first audio according to the second alignment path to obtain a second audio;

[0084] Introduce the dynamics of the reference voice into the second audio using the RMS energy value to obtain a merged audio;

[0085] Add reverberation to the merged audio through room impulse response convolution to obtain the final audio.

[0086] The third aspect of the present invention provides a computer-readable storage medium. The computer-readable storage medium includes a method program for aligning and merging multi-voice audio. When the method program for aligning and merging multi-voice audio is executed by a processor, the steps of the method for aligning and merging multi-voice audio are implemented.

[0087] In several embodiments provided by the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed with each other can be through some interfaces. The indirect coupling or communication connection of the devices or units can be electrical, mechanical, or other forms.

[0088] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0089] In addition, in each embodiment of the present invention, each functional unit may be entirely integrated in a processing unit, or each unit may be separately regarded as a unit, or two or more units may be integrated in one unit; the above-mentioned integrated unit may be implemented in the form of hardware, or in the form of a combination of hardware and software functional units.

[0090] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps including those of the above method embodiments; and the aforementioned storage medium includes: removable storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks or optical discs and other various media that can store program codes.

[0091] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiments of the present invention, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in the various embodiments of the present invention. And the aforementioned storage medium includes: removable storage devices, ROM, RAM, magnetic disks or optical discs and other various media that can store program codes.

Claims

1. A method for aligning and merging multi-voice audio, characterized in that It includes the following steps: Obtain the performance audio of different voices in the music score, and detect and extract the piano passages included in each said performance audio; Divide the sheet music by voice parts and extract the score for each voice part. Calculate the reference speed v of the sheet music based on the playing duration of the audio for each voice part and the number of beats included in the score mean ; Align the audio of each voice with the music score at the reference speed to obtain a first alignment path, perform speed change adjustment using a phase vocoder according to the first alignment path, and merge the speed-adjusted audios to obtain a first audio; Calculate the second alignment path between the reference voice and the music score, and perform non-linear speed adjustment on the first audio according to the second alignment path to obtain a second audio; Introduce the dynamics of the reference voice into the second audio using the RMS energy value to obtain a merged audio; Add reverberation to the merged audio through room impulse response convolution to obtain the final audio.

2. The method for aligning and merging multi-voice audio according to claim 1, characterized in that Divide the music score by voice and extract the score of each voice. The specific process of calculating the reference speed of the music score based on the performance duration of each voice audio and the number of beats included in the score is as follows: Divide the music score by voice parts to obtain the score s for each voice part n , analyze the duration of each section of audio for each voice part, that is, the duration of the piano section, parse out the number of beats contained in the music score corresponding to each piano section, and calculate the playing speed v of the current voice part based on the duration of the piano section and the number of beats it contains n , calculate the average value v of the playing speeds of all voice parts mean , obtain the average playing speed as the reference speed of the music score, that is: n represents the number of voice parts 3. The method for aligning and merging polyphonic audio according to claim 1, wherein Align the audio of each voice with the music score at the reference speed to obtain a first alignment path, perform speed change adjustment using a phase vocoder according to the first alignment path, and merge the speed-adjusted audios to obtain a first audio. The specific process is as follows: Divide the audio of each part into frames and the corresponding part's spectrum n Align to get the first alignment path d at the frame level n , align the audio of each part according to the first alignment path d n Use the phase vocoder to change the speed and get the same as v mean is the reference speed spectrum s n Consistent Audiox n , add the n parts of audio after speed adjustment and divide by n to obtain the first audio.

4. A method for aligning and merging multi-voice audio according to claim 3, characterized in that, When the scores of each part contain rests, first cut the audio and the scores of the part, and find the section where the audio contains the piano sound. The number of rests in the score is combined with the score reference speed v mean , calculate the total duration t of the rest, and fill in the corresponding length of the silent signal in The audio between the two parts is then used to obtain the audio b of the part that completes the rest duration. n , and then b n With sheet music n Align to obtain the first alignment path d n , and then the audio of each part is transmitted according to the first path d n Use the phase vocoder to change the speed and get the same as v mean The score for the reference tempo n Consistent Audiox n , and finally add the adjusted audios of the n parts and divide by n to obtain the merged first audio.

5. A method for aligning and merging multi-voice audio according to claim 1, characterized in that, Calculate the second alignment path between the reference voice and the music score, and perform non-linear speed adjustment on the first audio according to the second alignment path to obtain a second audio. The specific process is as follows: Select the audio of the reference voice part, and align the audio of the reference voice part with the musical score at the reference speed v mean to obtain the second alignment path; After non-linearly changing the speed of the first audio using a phase vocoder according to the second alignment path, a new second audio with the performance speed of the reference voice can be obtained.

6. A method for aligning and merging multi-voice audio according to claim 1, characterized in that, Introduce the dynamics of the reference voice into the second audio using the RMS energy value to obtain a merged audio. The specific process is as follows: Frame the audio signal of the reference voice according to a preset sampling rate, and calculate the RMS energy value of the reference voice audio frame by frame. Calculate the energy value of each voice audio in the second audio, and scale the energy value of each frame of each voice audio in the second audio to be consistent with the RMS energy value of each frame of the reference voice audio; Add the scaled voice audios in the second audio to obtain a merged audio.

7. A method for aligning and merging multi-voice audio according to claim 6, characterized in that The RMS energy of the i-th frame is expressed as: N represents the number of sampling points per frame, and x i represents the amplitude value of each sampling point of the signal.

8. A method for aligning and merging multi-voice audio according to claim 1, characterized in that Add reverberation to the merged audio through room impulse response convolution to obtain the final audio. The specific process is as follows: Select the room impulse response that meets the preset conditions, and convolve the room impulse response with the merged audio to obtain the final audio D final : D final = Dvv * RIR Dvv represents the merged audio, and RIR represents the room impulse response.

9. A system for aligning and merging multi-voice audio, characterized in that, The system includes: a memory and a processor. The memory includes a method program for aligning and merging multi-voice audio. When the method program for aligning and merging multi-voice audio is executed by the processor, the following steps are implemented: Obtain the performance audio of different voices in the music score, and detect and extract the piano passages included in each said performance audio; Divide the sheet music by voice parts and extract the score for each voice part. Calculate the reference tempo v of the sheet music based on the playing duration of the audio for each voice part and the number of beats included in the score for that voice part mean ; Align the audio of each voice with the music score at the reference speed to obtain a first alignment path, perform speed change adjustment using a phase vocoder according to the first alignment path, and merge the speed-adjusted audios to obtain a first audio; Calculate the second alignment path between the reference voice and the music score, and perform non-linear speed adjustment on the first audio according to the second alignment path to obtain a second audio; Introduce the dynamics of the reference voice into the second audio using the RMS energy value to obtain a merged audio; Add reverberation to the merged audio through room impulse response convolution to obtain the final audio.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a method program for aligning and merging polyphonic audio. When the method program for aligning and merging polyphonic audio is executed by a processor, the steps of a method for aligning and merging polyphonic audio as described in any one of claims 1 to 8 are implemented.

Images