Accompaniment adjustment method, device, equipment and storage medium

Abstract

The present invention belongs to the field of audio processing technology, and discloses an accompaniment adjustment method, device, equipment and storage medium. The method includes: when a user sings, determining the segment number of the current accompaniment; obtaining the accompaniment feature information of the current accompaniment, the singing feature information of the user and the historical singing feature information; obtaining accompaniment adjustment information based on the accompaniment feature information of the current accompaniment, the singing feature information of the user and the historical singing feature information; adjusting the segment accompaniment information of the next segment based on the accompaniment adjustment information and the segment number. Through the above method, the accompaniment adjustment information can be obtained based on the accompaniment feature information of the current accompaniment, the singing feature information of the user and the historical singing feature information, so as to adjust the segment accompaniment information of the next segment sung by the user in real time, thereby realizing real-time adjustment of the accompaniment, and then making the adjusted accompaniment more suitable for the user, improving the user's singing effect.

Description

Technical Field

[0001] The present invention relates to the technical field of audio processing, and in particular to an accompaniment adjustment method, device, equipment and storage medium. Background Art

[0002] The current accompaniment adjustment method generally involves adjusting the sound after the user has finished singing. This can basically ensure the correctness of pitch and volume, but there is a possibility of timbre distortion. Most importantly, it is time-consuming and requires waiting for audio processing, resulting in a poor customer experience.

[0003] The above content is only used to assist in understanding the technical solution of the present invention and does not constitute an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of the present invention is to provide an accompaniment adjustment method, device, equipment and storage medium, aiming to solve the technical problem that the existing technology cannot adjust the accompaniment in real time, resulting in poor user experience.

[0005] To achieve the above object, the present invention provides an accompaniment adjustment method, which comprises the following steps:

[0006] When the user sings, determine the segment number of the current accompaniment;

[0007] Acquiring accompaniment feature information of the current accompaniment, singing feature information of the user, and historical singing feature information;

[0008] Obtaining accompaniment adjustment information according to the accompaniment feature information of the current accompaniment, the singing feature information of the user, and the historical singing feature information;

[0009] The segment accompaniment information of the next segment is adjusted according to the accompaniment adjustment information and the segment number.

[0010] Optionally, before determining the segment number of the current accompaniment, the method further includes:

[0011] Get the accompaniment beat information of the current accompaniment;

[0012] According to the accompaniment beat information, the current accompaniment is segmented according to the accompaniment beat to obtain a plurality of accompaniment segments;

[0013] The plurality of accompaniment segments are numbered to obtain segment numbers.

[0014] Optionally, obtaining the accompaniment adjustment information according to the accompaniment feature information of the current accompaniment, the user's singing feature information, and the historical singing feature information includes:

[0015] Accompaniment adjustment information is obtained according to the accompaniment feature information of the current accompaniment, the singing feature information of the user, the historical singing feature information, the segment accompaniment information of the next segment and the pre-trained accompaniment adjustment model.

[0016] Optionally, the training process of the accompaniment adjustment model includes:

[0017] Obtaining an input feature set based on the accompaniment feature information, the singing feature information, and the historical singing feature information;

[0018] Constructing a multivariate function fitting model, and using the multivariate function fitting model as a model to be trained;

[0019] The model to be trained is trained according to the input feature set to obtain an accompaniment adjustment model.

[0020] Optionally, obtaining the input feature set according to the accompaniment feature information, the singing feature information and the historical singing feature information includes:

[0021] Obtaining, according to the accompaniment feature information, a segmented singing score, a segmented loudness feature, a segmented timbre feature, and a segmented pitch feature of each segmented accompaniment;

[0022] Obtaining singing loudness characteristics, singing timbre characteristics and singing pitch characteristics of the user singing each of the segmented accompaniments according to the singing characteristic information;

[0023] Obtaining historical singing loudness features, historical singing timbre features, and historical singing pitch features of the user's historical singing of the current accompaniment according to the historical singing feature information;

[0024] The segmented singing scores, segmented loudness features, segmented timbre features, segmented pitch features, singing loudness features, singing timbre features, singing pitch features, historical singing loudness features, historical singing timbre features and historical singing pitch features are used as input feature sets.

[0025] Optionally, obtaining the accompaniment adjustment information according to the accompaniment feature information of the current accompaniment, the singing feature information of the user, the historical singing feature information, the accompaniment information of the next segment and the pre-trained accompaniment adjustment model includes:

[0026] Inputting the accompaniment feature information of the current accompaniment, the singing feature information of the user, and the historical singing feature information into the accompaniment adjustment model to obtain a loudness sorting result, a pitch sorting result, and a timbre sorting result;

[0027] determining an optimal loudness matching degree according to the loudness sorting result;

[0028] determining the best pitch matching degree according to the pitch sorting result;

[0029] Determining the best timbre matching degree according to the timbre sorting result;

[0030] Accompaniment adjustment information is obtained according to the best loudness matching degree, the best pitch matching degree, the best timbre matching degree and the segment accompaniment information of the next segment.

[0031] Optionally, before determining the segment number of the current accompaniment, the method further includes:

[0032] When the user starts playing the initial accompaniment, determining the beginning segment accompaniment according to the segment number of the initial accompaniment;

[0033] Obtaining opening accompaniment feature information according to the opening segment accompaniment;

[0034] Inputting historical singing feature information into the accompaniment adjustment model to obtain the opening segment adjustment information;

[0035] The opening accompaniment characteristic information is adjusted according to the opening segment adjustment information to obtain the current accompaniment.

[0036] In addition, to achieve the above-mentioned object, the present invention further provides an accompaniment adjustment device, the accompaniment adjustment device comprising:

[0037] The determination module is used to determine the segment number of the current accompaniment when the user sings;

[0038] An acquisition module, configured to acquire accompaniment feature information of a current accompaniment, singing feature information of the user, and historical singing feature information;

[0039] a processing module, configured to obtain accompaniment adjustment information according to the accompaniment characteristic information of the current accompaniment, the singing characteristic information of the user, and the historical singing characteristic information;

[0040] An adjustment module is used to adjust the segment accompaniment information of the next segment according to the accompaniment adjustment information and the segment number.

[0041] In addition, to achieve the above-mentioned purpose, the present invention also proposes an accompaniment adjustment device, which includes: a memory, a processor, and an accompaniment adjustment program stored in the memory and executable on the processor, wherein the accompaniment adjustment program is configured to implement the steps of the accompaniment adjustment method described above.

[0042] In addition, to achieve the above-mentioned purpose, the present invention further proposes a storage medium, on which an accompaniment adjustment program is stored. When the accompaniment adjustment program is executed by a processor, the steps of the accompaniment adjustment method described above are implemented.

[0043] When a user sings, the present invention determines the segment number of the current accompaniment; obtains accompaniment feature information of the current accompaniment, the user's singing feature information, and historical singing feature information; obtains accompaniment adjustment information based on the accompaniment feature information of the current accompaniment, the user's singing feature information, and historical singing feature information; and adjusts the segment accompaniment information of the next segment based on the accompaniment adjustment information and the segment number. In this way, the accompaniment adjustment information can be obtained based on the accompaniment feature information of the current accompaniment, the user's singing feature information, and historical singing feature information, thereby adjusting the segment accompaniment information of the next segment sung by the user in real time, thereby achieving real-time adjustment of the accompaniment, thereby making the adjusted accompaniment more suitable for the user and improving the user's singing effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a structural diagram of an accompaniment adjustment device in a hardware operating environment according to an embodiment of the present invention;

[0045] Figure 2 Schematic diagram of the flow of the first embodiment of the accompaniment adjustment method of the present invention;

[0046] Figure 3 2 is a flow chart of a second embodiment of the accompaniment adjustment method of the present invention;

[0047] Figure 4 2 is a flow chart of a third embodiment of the accompaniment adjustment method of the present invention;

[0048] Figure 5 This is a structural block diagram of the first embodiment of the accompaniment adjustment device of the present invention.

[0049] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0050] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0051] Reference Figure 1 , Figure 1 This is a schematic diagram of the structure of an accompaniment adjustment device in a hardware operating environment according to an embodiment of the present invention.

[0052] like Figure 1As shown, the accompaniment adjustment device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to implement connection and communication between these components. The user interface 1003 may include a display screen (Display) and an input unit such as a keyboard (Keyboard). Optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a wireless fidelity (Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (RAM) memory or a stable non-volatile memory (NVM), such as a disk storage. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0053] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the accompaniment adjustment device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0054] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module and an accompaniment adjustment program.

[0055] exist Figure 1 In the accompaniment adjustment device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the accompaniment adjustment device of the present invention can be set in the accompaniment adjustment device, and the accompaniment adjustment device calls the accompaniment adjustment program stored in the memory 1005 through the processor 1001 and executes the accompaniment adjustment method provided by the embodiment of the present invention.

[0056] The embodiment of the present invention provides an accompaniment adjustment method, referring to Figure 2 , Figure 2 FIG. 4 is a flow chart of a first embodiment of an accompaniment adjustment method according to the present invention.

[0057] In this embodiment, the accompaniment adjustment method includes the following steps:

[0058] Step S10: When the user sings, determine the segment number of the current accompaniment.

[0059] It should be noted that the execution entity of this embodiment is a server, which can be a physical server, a cloud server, or other servers that can implement this function, and this embodiment does not impose any restrictions on this.

[0060] It should be understood that current singing assistance software generally adjusts the accompaniment of a song based on the collected user's singing voice after the user completes the singing of the song, and finally obtains the adjusted work. However, it cannot adjust the accompaniment in real time during the user's singing process to improve the user's singing effect and singing performance. The solution of this embodiment adjusts the accompaniment information of the next segment of the current accompaniment sung by the user in real time based on the accompaniment feature information, the user's singing feature information, and historical singing information, so that the accompaniment of the next segment can be more suitable for the current user, thereby improving the user's singing effect and singing performance.

[0061] In a specific implementation, when a user sings, determining the segment number of the current accompaniment means: when the singing sound information input by the user through the microphone while the accompaniment is playing is detected, the specific segment of the current accompaniment that the user starts singing is determined based on the singing sound information. The detection is completed in real time, that is, the detection and determination of the segment number and segment accompaniment information of the current accompaniment are performed while the user sings the accompaniment song. The singing sound information refers to the sound information of the user singing the accompaniment, and does not include other environmental noises and other sounds of the user unrelated to the song being sung.

[0062] It should be noted that the segment number refers to the number of each segment accompaniment obtained by segmenting the current accompaniment according to the accompaniment beat.

[0063] Furthermore, in order to accurately segment the accompaniment, before step S10, it also includes: obtaining the accompaniment beat information of the current accompaniment; segmenting the current accompaniment according to the accompaniment beat information to obtain several accompaniment segments; numbering the several accompaniment segments to obtain segment numbers.

[0064] In a specific implementation, the accompaniment beat information includes but is not limited to information related to the accompaniment music beat of the current accompaniment, and other information related to the accompaniment.

[0065] It should be noted that segmenting the current accompaniment according to the accompaniment beat information to obtain a plurality of segmented accompaniments means segmenting the current accompaniment according to the accompaniment beat information in chronological order and according to the accompaniment beat information. Specifically, the accompaniment may be divided into the following parts: prelude, verse, main-chorus transition section, chorus, bridge or interlude, and outing. Other parts may also be included, and this embodiment does not limit this. The resulting segments representing the different accompaniment beats are the segmented accompaniments.

[0066] It should be understood that numbering the several fragment accompaniments to obtain fragment numbers means that after the fragment accompaniments are obtained, all the fragment accompaniments are numbered in the order of the current accompaniment, for example: the fragment accompaniment representing the prelude is No. 1, the fragment accompaniment representing the main song is No. 2, and the remaining fragment accompaniments are numbered in sequence.

[0067] In this way, the current accompaniment of an entire section is divided into segments and numbered, and several segment accompaniments with segment numbers are obtained. The changes in the accompaniment tone often occur at the connection between the parts, that is, in the same part, the tone of the accompaniment is not much different, and there are tone differences between different parts. Then, the accompaniment is adjusted in each segment accompaniment, making it more flexible to adjust the accompaniment in real time to match the user's singing.

[0068] Step S20: Acquire the accompaniment feature information of the current accompaniment, the user's singing feature information, and historical singing feature information.

[0069] In a specific implementation, the accompaniment feature information includes information related to the singing scores of all users who have selected and sung the current accompaniment, as well as information related to the pitch, timbre and loudness of the accompaniment.

[0070] It should be noted that the singing feature information refers to the scoring information of each segment when the user sings the current accompaniment, as well as related information such as the singing pitch, timbre and loudness.

[0071] It should be understood that the historical singing feature information includes but is not limited to the historical scoring information of all accompaniments sung by the current user, as well as related information such as the loudness, timbre and pitch of the historical singing.

[0072] Step S30: Obtaining accompaniment adjustment information according to the accompaniment characteristic information of the current accompaniment, the user's singing characteristic information and the historical singing characteristic information.

[0073] In a specific implementation, obtaining accompaniment adjustment information based on the accompaniment feature information of the current accompaniment, the singing feature information of the user, and the historical singing feature information means: training the model to be trained based on the accompaniment feature information of the current accompaniment, the singing feature information of the user, and the historical singing feature information to obtain a pitch adjustment model, and then inputting the singing feature information into the accompaniment adjustment model to obtain the accompaniment adjustment information.

[0074] It should be noted that the accompaniment adjustment information refers to the relevant information for adjusting the pitch, timbre and loudness of the accompaniment of the next segment, that is, the adjustment basis for modifying the segment accompaniment information of the next segment.

[0075] Step S40: adjusting the segment accompaniment information of the next segment according to the accompaniment adjustment information and the segment number.

[0076] It should be understood that adjusting the segment accompaniment information of the next segment according to the accompaniment adjustment information and the segment number means: determining the segment number of the next segment according to the segment number, that is, specifically determining which segment accompaniment the next segment is, and then adjusting the timbre, loudness and pitch in the segment accompaniment information of the next segment according to the accompaniment adjustment information, thereby realizing real-time adjustment of the accompaniment of the next segment according to the singing voice of the user singing each segment.

[0077] This embodiment determines the segment number of the current accompaniment when the user sings; obtains the accompaniment feature information of the current accompaniment, the user's singing feature information, and historical singing feature information; obtains accompaniment adjustment information based on the accompaniment feature information of the current accompaniment, the user's singing feature information, and historical singing feature information; and adjusts the segment accompaniment information of the next segment based on the accompaniment adjustment information and the segment number. In this way, the accompaniment adjustment information can be obtained based on the accompaniment feature information of the current accompaniment, the user's singing feature information, and historical singing feature information, thereby adjusting the segment accompaniment information of the next segment sung by the user in real time, thereby achieving real-time adjustment of the accompaniment, thereby making the adjusted accompaniment more suitable for the user and improving the user's singing effect.

[0078] refer to Figure 3 , Figure 3 FIG. 4 is a flow chart of a second embodiment of an accompaniment adjustment method according to the present invention.

[0079] Based on the first embodiment described above, the accompaniment adjustment method of this embodiment includes, in step S30:

[0080] Step S301: Accompaniment adjustment information is obtained according to the accompaniment feature information of the current accompaniment, the singing feature information of the user, the historical singing feature information, the segment accompaniment information of the next segment and the pre-trained accompaniment adjustment model.

[0081] It should be noted that the model to be trained refers to a pre-established untrained model, and the accompaniment adjustment model is obtained by training the model to be trained.

[0082] It should be understood that the accompaniment adjustment model refers to a model that can directly output the accompaniment adjustment information for adjusting the next segment after inputting the singing characteristic information of the user's singing, especially the singing characteristic information of the current segment of the current accompaniment.

[0083] It should be noted that, based on the accompaniment feature information of the current accompaniment, the singing feature information of the user, the historical singing feature information, the fragment accompaniment information of the next segment and the accompaniment adjustment model, obtaining the accompaniment adjustment information means inputting the accompaniment feature information, the singing feature information and the historical singing feature information into the trained accompaniment adjustment model, and then obtaining the accompaniment adjustment information based on the output of the accompaniment adjustment model and the fragment accompaniment information of the next segment.

[0084] Furthermore, in order to obtain the accompaniment adjustment model, step S301 includes: obtaining an input feature set based on the accompaniment feature information, singing feature information and historical singing feature information; constructing a multivariate function fitting model, and using the multivariate function fitting model as the model to be trained; training the model to be trained according to the input feature set to obtain the accompaniment adjustment model.

[0085] In a specific implementation, obtaining an input feature set based on the accompaniment feature information, singing feature information and historical singing feature information means: extracting and obtaining the parameters related to timbre, loudness, pitch and score in the accompaniment feature information, singing feature information and historical singing feature information and using them as the input feature set for training the model to be trained.

[0086] It should be noted that the multivariate function fitting model refers to a multivariate function fitting model of a first-order linear multiplication + second-order nonlinear combination, which is then used as the model to be trained. The formula of the model to be trained is:

[0087]

[0088] Where η is the model parameter, and α, β, S, A, T, and P are the input feature sets of the model input.

[0089] Furthermore, in order to obtain an accurate input feature set, the step of obtaining the input feature set based on the accompaniment feature information, singing feature information and historical singing feature information includes: obtaining the segmented singing score, segmented loudness feature, segmented timbre feature and segmented pitch feature of each segmented accompaniment based on the accompaniment feature information; obtaining the singing loudness feature, singing timbre feature and singing pitch feature of the user singing each segmented accompaniment based on the singing feature information; obtaining the historical singing loudness feature, historical singing timbre feature and historical singing pitch feature of the user's historical singing of the current accompaniment based on the historical singing feature information; and using the segmented singing score, segmented loudness feature, segmented timbre feature, segmented pitch feature, singing loudness feature, singing timbre feature, singing pitch feature, historical singing loudness feature, historical singing timbre feature and historical singing pitch feature as the input feature set.

[0090] It should be understood that obtaining the segmented singing score, segmented loudness feature, segmented timbre feature and segmented pitch feature of each segmented accompaniment according to the accompaniment feature information means extracting the relevant segmented singing score, segmented loudness feature, segmented timbre feature and segmented pitch feature according to the accompaniment feature information. Specifically, the steps for obtaining the average singing score are: obtaining the highest singing score S after the accompaniment is selected by the user max , the lowest score is S min , average score S avg , and score decile values ​​S1, S2, S3, S4, S5, S6, S7, S8, S9, S 10 Segmental loudness characteristics: the maximum amplitude β of the sound wave vibration max X , minimum value of the acoustic vibration amplitude Average value of sound wave vibration amplitude Quartile values ​​of acoustic vibration amplitude Segmental timbre characteristics: including the vibration waveform of the sound wave and the tenth value of the spectrum structure Segmental pitch characteristics: the maximum frequency of the sound wave Minimum frequency Frequency average Frequency quartiles

[0091] In a specific implementation, the singing loudness features, singing timbre features and singing pitch features of the user's singing of each segmented accompaniment are obtained according to the singing feature information, and feature collection is performed in accompaniment segment units, with a sampling frequency of 44.1KHz, a sampling bit depth of 24bit, an ID of the song with accompaniment, an overall singing score, a singing segment k, and 24 features for each segment, including singing loudness features, singing timbre features and singing pitch features.

[0092] It should be noted that the singing loudness characteristic refers to the maximum amplitude α of the sound wave vibration max X , minimum value of the acoustic vibration amplitude Average value of sound wave vibration amplitude Quartile values ​​of acoustic vibration amplitude

[0093] It should be understood that the singing timbre characteristics refer to: the vibration waveform of each sound wave, the tenth value of the spectrum structure

[0094] In specific implementation, the singing pitch feature refers to: the maximum frequency of the sound wave Minimum frequency Frequency average Frequency quartiles

[0095] In a specific implementation, obtaining the historical loudness, timbre, and pitch characteristics of the user's historical singing of the current accompaniment based on the historical singing feature information means: if the user's historical karaoke score is a karaoke score, and the standard accompaniment segment number is k, then recording the time-sequential loudness settings of each segment: A1, A2, A3...AN; recording the time-sequential timbre settings of each segment: T1, T2, T3...TN; and recording the time-sequential pitch settings of each segment: P1, D2, D3...PN. If N is 1000, the loudness, timbre, and pitch adjustment points in the standard accompaniment segment are set to 1000 points, the loudness A value range is 0-100dB, the timbre T value range is 20-20000Hz, and the timbre P is adjusted in an enumerated value manner, with 10 preset timbre adjustment modes.

[0096] It should be noted that when inputting the input feature set into the model to be trained, the pitch feature For example, the maximum value is collected during the collection Minimum frequency Frequency average Frequency quartiles There are 7 values ​​in total. Here is the formula i takes values ​​from 1 to 7, that is, including Other parameters and Similar, no further description is given. In addition, β in this step i is a vector containing Other parameters and β i Similar, no further elaboration.

[0097] Secondly, the model discrimination target is determined to be the karaoke score, that is, the singing score. The historical singing score with accompaniment is scored from 0 to 100 points and mapped to the [0,1] interval. The decision function is:

[0098]

[0099] Among them, score = 1, and the threshold is selected as 0.8, that is, the singing with accompaniment score of 80 points or above is a positive example, and the historical karaoke score below 80 points is a negative example. The singing and accompaniment are restricted to belong to the same song ID, so as to construct a sample set and train an accompaniment adjustment model with the ability to fit nonlinear features and deep learning capabilities.

[0100] In this way, it is possible to obtain specific singing scores, loudness, pitch and timbre parameters based on the accompaniment feature information, singing feature information and historical singing feature information, making the input feature set more accurate and comprehensive, and thus making the calculation of the trained accompaniment adjustment model more accurate.

[0101] Furthermore, in order to obtain accompaniment adjustment information more accurately and effectively, thereby improving the user experience and adjusting the accompaniment in real time, step S301 includes: inputting the accompaniment feature information of the current accompaniment, the user's singing feature information and the historical singing feature information into the accompaniment adjustment model to obtain loudness sorting results, pitch sorting results and timbre sorting results; determining the best loudness matching according to the loudness sorting results; determining the best pitch matching according to the pitch sorting results; determining the best timbre matching according to the timbre sorting results; obtaining accompaniment adjustment information according to the best loudness matching, best pitch matching, best timbre matching and the accompaniment information of the next segment.

[0102] It should be understood that the loudness matching, pitch matching, and timbre matching refer to the matching of the loudness, pitch, and timbre corresponding to each accompaniment segment after inputting the accompaniment feature information, performance feature information, and historical performance information into the accompaniment adjustment model. In other words, the matching of all loudness values ​​to each accompaniment segment, and the same applies to pitch and timbre. Because the performance feature information is updated in real time as the user performs, the loudness matching, pitch matching, and timbre matching output by the accompaniment adjustment model are also constantly changing.

[0103] In a specific implementation, the accompaniment feature information of the current accompaniment, the singing feature information of the user, and the historical singing feature information are input into the accompaniment adjustment model to obtain the loudness sorting result, the pitch sorting result, and the timbre sorting result. This means that all the loudness matching degrees, pitch matching degrees, and timbre matching degrees are output in the accompaniment adjustment model and sorted according to the matching degree, and the loudness sorting result corresponding to the loudness matching value, the timbre sorting result corresponding to the timbre matching value, and the tone sorting result corresponding to the pitch matching value are obtained respectively.

[0104] It should be understood that determining the best loudness matching degree according to the loudness sorting results, determining the best pitch matching degree according to the pitch sorting results, and determining the best timbre matching degree according to the timbre sorting results means: taking the loudness matching degree, pitch matching degree, and timbre matching degree that are ranked first among the loudness sorting results, pitch sorting results, and pitch sorting results, i.e., the corresponding loudness matching degree with the highest matching degree, as the best loudness matching degree, best pitch matching degree, and best timbre matching degree.

[0105] It should be noted that obtaining the accompaniment adjustment information based on the best loudness matching, the best pitch matching, the best timbre matching and the segment accompaniment information of the next segment means: according to the best loudness matching, the best pitch matching and the best timbre matching and combined with the segment accompaniment information of the next segment, determining the optimal values ​​of the segment loudness characteristics, segment timbre characteristics and segment pitch characteristics of the next segment accompaniment as the accompaniment adjustment information.

[0106] In this way, the next segment can be accurately obtained according to the information output by the accompaniment adjustment model, that is, the best loudness matching, best pitch matching and best timbre matching of the next segment accompaniment, thereby obtaining the accompaniment adjustment information, and then the accompaniment of the next segment can be adjusted to the most suitable for the current user's singing, thereby improving the user's singing effect and usage experience.

[0107] This embodiment obtains accompaniment adjustment information based on the accompaniment feature information of the current accompaniment, the user's singing feature information, historical singing feature information, the next segmented accompaniment information, and a pre-trained accompaniment adjustment model. In this way, it is possible to obtain the accompaniment adjustment model based on the accompaniment feature information, the singing feature information, and the historical singing feature information, and then obtain the accompaniment adjustment information based on the singing feature information and the accompaniment adjustment model, so that the real-time adjustment of the accompaniment being sung by the user is more accurate and comprehensive, and the accompaniment can be adjusted to the state that best suits the user more quickly, thereby improving the user's singing effect and user experience.

[0108] refer to Figure 4 , Figure 4 FIG. 4 is a flow chart of a third embodiment of an accompaniment adjustment method according to the present invention.

[0109] Based on the first embodiment, the accompaniment adjustment method of this embodiment further includes, before step S10:

[0110] Step S101: When the user starts playing the initial accompaniment, the beginning segment accompaniment is determined according to the segment number of the initial accompaniment.

[0111] It should be noted that the initial accompaniment refers to the unadjusted accompaniment after the user selects the song and starts playing.

[0112] It should be understood that when the user starts playing the initial accompaniment, determining the beginning segment accompaniment according to the segment number of the initial accompaniment means: when it is detected that the user starts playing the initial accompaniment, determining the beginning segment accompaniment according to the segment number of the initial accompaniment as the beginning segment accompaniment.

[0113] Step S102: obtaining opening accompaniment feature information according to the opening segment accompaniment.

[0114] In a specific implementation, obtaining the opening accompaniment feature information according to the opening segment accompaniment means searching the accompaniment feature information for feature information of pitch, timbre and loudness corresponding to the opening segment accompaniment as the opening accompaniment feature information.

[0115] Step S103: inputting the historical singing feature information into the accompaniment adjustment model to obtain the opening segment adjustment information.

[0116] It should be noted that inputting the historical singing feature information into the accompaniment adjustment model to obtain the opening segment adjustment information means: at this time the user has not started singing the accompaniment song, but needs to adjust the opening segment accompaniment, so the historical singing feature information is input into the accompaniment adjustment model to obtain the adjustment information for the opening segment accompaniment, that is, the opening segment adjustment information.

[0117] Step S104: adjusting the opening accompaniment characteristic information according to the opening segment adjustment information to obtain the current accompaniment.

[0118] It should be understood that adjusting the opening accompaniment characteristic information according to the opening segment adjustment information to obtain the current accompaniment means modifying the loudness, timbre and pitch characteristic information in the opening accompaniment characteristic information of the opening segment accompaniment according to the opening segment adjustment information, and the accompaniment after the modification is completed is the current accompaniment.

[0119] In this embodiment, when the user begins playing the initial accompaniment, the beginning segment accompaniment is determined based on the segment number of the initial accompaniment; the beginning accompaniment feature information of the beginning segment accompaniment is obtained based on the accompaniment feature information; the historical singing feature information is input into the accompaniment adjustment model to obtain the beginning segment adjustment information; and the beginning accompaniment feature information of the beginning segment accompaniment is adjusted based on the beginning segment adjustment information to obtain the current accompaniment. In this way, the beginning of the first segment of the accompaniment can be adjusted based on the user's historical singing feature information before the user begins singing the accompaniment, so that the beginning segment accompaniment is most suitable for the user's singing, thereby improving the user's user experience and improving the user's singing effect and performance.

[0120] In addition, an embodiment of the present invention further provides a storage medium on which an accompaniment adjustment program is stored. When the accompaniment adjustment program is executed by a processor, the steps of the accompaniment adjustment method described above are implemented.

[0121] Since the storage medium adopts all the technical solutions of all the above embodiments, it has at least all the beneficial effects brought by the technical solutions of the above embodiments, which will not be described one by one here.

[0122] Reference Figure 5 , Figure 5 This is a structural block diagram of the first embodiment of the accompaniment adjustment device of the present invention.

[0123] like Figure 5 As shown, the accompaniment adjustment device proposed in the embodiment of the present invention includes:

[0124] The determination module 10 is used to determine the segment number of the current accompaniment when the user sings.

[0125] The acquisition module 20 is used to acquire the accompaniment feature information of the current accompaniment, the singing feature information of the user, and the historical singing feature information.

[0126] The processing module 30 is used to obtain accompaniment adjustment information according to the accompaniment feature information of the current accompaniment, the user's singing feature information and the historical singing feature information.

[0127] The adjustment module 40 is configured to adjust the accompaniment information of the next segment according to the accompaniment adjustment information and the segment number.

[0128] This embodiment determines the segment number of the current accompaniment when the user sings; obtains the accompaniment feature information of the current accompaniment, the user's singing feature information, and historical singing feature information; obtains accompaniment adjustment information based on the accompaniment feature information of the current accompaniment, the user's singing feature information, and historical singing feature information; and adjusts the segment accompaniment information of the next segment based on the accompaniment adjustment information and the segment number. In this way, the accompaniment adjustment information can be obtained based on the accompaniment feature information of the current accompaniment, the user's singing feature information, and historical singing feature information, thereby adjusting the segment accompaniment information of the next segment sung by the user in real time, thereby achieving real-time adjustment of the accompaniment, thereby making the adjusted accompaniment more suitable for the user and improving the user's singing effect.

[0129] In one embodiment, the determination module 10 is further configured to obtain accompaniment beat information of the current accompaniment; segment the current accompaniment according to the accompaniment beat information to obtain a plurality of accompaniment segments; and number the plurality of accompaniment segments to obtain segment numbers.

[0130] In one embodiment, the processing module 30 is further used to obtain accompaniment adjustment information based on the accompaniment feature information of the current accompaniment, the user's singing feature information, historical singing feature information, the next segment accompaniment information and a pre-trained accompaniment adjustment model.

[0131] In one embodiment, the processing module 30 is also used to obtain an input feature set based on the accompaniment feature information, singing feature information and historical singing feature information; construct a multivariate function fitting model, and use the multivariate function fitting model as the model to be trained; train the model to be trained according to the input feature set to obtain an accompaniment adjustment model.

[0132] In one embodiment, the processing module 30 is further used to obtain the segmented singing score, segmented loudness feature, segmented timbre feature and segmented pitch feature of each segmented accompaniment according to the accompaniment feature information; obtain the singing loudness feature, singing timbre feature and singing pitch feature of each segmented accompaniment sung by the user according to the singing feature information; obtain the historical singing loudness feature, historical singing timbre feature and historical singing pitch feature of the user's historical singing of the current accompaniment according to the historical singing feature information; and use the segmented singing score, segmented loudness feature, segmented timbre feature, segmented pitch feature, singing loudness feature, singing timbre feature, singing pitch feature, historical singing loudness feature, historical singing timbre feature and historical singing pitch feature as the input feature set.

[0133] In one embodiment, the processing module 30 is further configured to input the accompaniment feature information of the current accompaniment, the singing feature information of the user, and the historical singing feature information into the accompaniment adjustment model to obtain a loudness sorting result, a pitch sorting result, and a timbre sorting result; determine an optimal loudness matching degree based on the loudness sorting result; determine an optimal pitch matching degree based on the pitch sorting result; determine an optimal timbre matching degree based on the timbre sorting result; and obtain accompaniment adjustment information based on the optimal loudness matching degree, the optimal pitch matching degree, the optimal timbre matching degree, and the accompaniment information of the next segment.

[0134] In one embodiment, the determination module 10 also includes a preset module, which is used to determine the beginning segment accompaniment according to the segment number of the initial accompaniment when the user starts to play the initial accompaniment; obtain the beginning accompaniment feature information according to the beginning segment accompaniment; input the historical singing feature information into the accompaniment adjustment model to obtain the beginning segment adjustment information; adjust the beginning accompaniment feature information according to the beginning segment adjustment information to obtain the current accompaniment.

[0135] It should be understood that the above is only an example and does not constitute any limitation to the technical solution of the present invention. In specific applications, those skilled in the art can make settings as needed, and the present invention does not impose any limitation on this.

[0136] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present invention. In practical applications, technicians in this field can select part or all of it according to actual needs to achieve the purpose of the embodiment scheme, and no limitation is made here.

[0137] In addition, for technical details not fully described in this embodiment, reference can be made to the accompaniment adjustment method provided in any embodiment of the present invention, and will not be repeated here.

[0138] In addition, it should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0139] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0140] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, or of course by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as a read-only memory (ROM) / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0141] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for adjusting accompaniment, characterized in that: The accompaniment adjustment method comprises: When the user sings, determine the segment number of the current accompaniment; Acquiring accompaniment feature information of the current accompaniment, singing feature information of the user, and historical singing feature information; Accompaniment adjustment information is obtained based on the accompaniment feature information of the current accompaniment, the singing feature information of the user, and the historical singing feature information, wherein the accompaniment feature information includes the singing scores of all users singing the current accompaniment, the pitch, timbre, and loudness of the accompaniment; the singing feature information refers to the scoring information, the pitch, timbre, and loudness of each segment when the current user sings the current accompaniment; and the historical singing feature information includes the historical scoring information, the loudness, timbre, and pitch of all accompaniments sung by the current user; The segment accompaniment information of the next segment is adjusted according to the accompaniment adjustment information and the segment number.

2. The method according to claim 1, wherein Before determining the segment number of the current accompaniment, the method further includes: Get the accompaniment beat information of the current accompaniment; According to the accompaniment beat information, the current accompaniment is segmented according to the accompaniment beat to obtain a plurality of accompaniment segments; The plurality of accompaniment segments are numbered to obtain segment numbers.

3. The method according to claim 1, wherein The obtaining of accompaniment adjustment information according to the accompaniment feature information of the current accompaniment, the singing feature information of the user, and the historical singing feature information includes: Accompaniment adjustment information is obtained according to the accompaniment feature information of the current accompaniment, the singing feature information of the user, the historical singing feature information, the segment accompaniment information of the next segment and the pre-trained accompaniment adjustment model.

4. The method according to claim 3, wherein The training process of the accompaniment adjustment model includes: Obtaining an input feature set based on the accompaniment feature information, the singing feature information, and the historical singing feature information; Constructing a multivariate function fitting model, and using the multivariate function fitting model as a model to be trained; The model to be trained is trained according to the input feature set to obtain an accompaniment adjustment model.

5. The method according to claim 4, wherein The input feature set is obtained according to the accompaniment feature information, the singing feature information and the historical singing feature information, including: Obtaining, according to the accompaniment feature information, a segmented singing score, a segmented loudness feature, a segmented timbre feature, and a segmented pitch feature of each segmented accompaniment; Obtaining singing loudness characteristics, singing timbre characteristics and singing pitch characteristics of the user singing each of the segmented accompaniments according to the singing characteristic information; Obtaining historical singing loudness features, historical singing timbre features, and historical singing pitch features of the user's historical singing of the current accompaniment according to the historical singing feature information; The segmented singing scores, segmented loudness features, segmented timbre features, segmented pitch features, singing loudness features, singing timbre features, singing pitch features, historical singing loudness features, historical singing timbre features and historical singing pitch features are used as input feature sets.

6. The method according to claim 4, wherein The method of obtaining accompaniment adjustment information according to the accompaniment feature information of the current accompaniment, the singing feature information of the user, the historical singing feature information, the accompaniment information of the next segment and the pre-trained accompaniment adjustment model includes: Inputting the accompaniment feature information of the current accompaniment, the singing feature information of the user, and the historical singing feature information into the accompaniment adjustment model to obtain a loudness sorting result, a pitch sorting result, and a timbre sorting result; determining an optimal loudness matching degree according to the loudness sorting result; determining the best pitch matching degree according to the pitch sorting result; Determining the best timbre matching degree according to the timbre sorting result; Accompaniment adjustment information is obtained according to the best loudness matching degree, the best pitch matching degree, the best timbre matching degree and the segment accompaniment information of the next segment.

7. The method according to any one of claims 1 to 6, characterized in that Before determining the segment number of the current accompaniment, the method further includes: When the initial accompaniment starts to be played, the beginning segment accompaniment is determined according to the segment number of the initial accompaniment; Obtaining opening accompaniment feature information according to the opening segment accompaniment; Inputting historical singing feature information into the accompaniment adjustment model to obtain the opening segment adjustment information; The opening accompaniment characteristic information is adjusted according to the opening segment adjustment information to obtain the current accompaniment.

8. An accompaniment adjustment device, characterized in that: The accompaniment adjustment device comprises: The determination module is used to determine the segment number of the current accompaniment when the user sings; An acquisition module, configured to acquire accompaniment feature information of a current accompaniment, singing feature information of the user, and historical singing feature information; a processing module, configured to obtain accompaniment adjustment information based on accompaniment feature information of the current accompaniment, singing feature information of the user, and historical singing feature information, wherein the accompaniment feature information includes singing scores of all users singing the current accompaniment, the pitch, timbre, and loudness of the accompaniment; the singing feature information refers to the scoring information, pitch, timbre, and loudness of each segment when the current user sings the current accompaniment; and the historical singing feature information includes historical scoring information, loudness, timbre, and pitch of all accompaniments sung by the current user; An adjustment module is used to adjust the segment accompaniment information of the next segment according to the accompaniment adjustment information and the segment number.

9. An accompaniment adjustment device, characterized in that: The device comprises: a memory, a processor, and an accompaniment adjustment program stored in the memory and executable on the processor, wherein the accompaniment adjustment program is configured to implement the accompaniment adjustment method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores an accompaniment adjustment program, which, when executed by the processor, implements the accompaniment adjustment method according to any one of claims 1 to 7.

Images