Harmony processing method, device, equipment and medium

Abstract

Embodiments of the present disclosure relate to a harmony processing method, apparatus, device, and medium. The method includes: in response to a triggering operation on a target harmony control, obtaining a harmony interval corresponding to the target harmony control; performing pitch shifting processing on a first sound of the original input according to the harmony interval to obtain a second sound; wherein the interval between the first sound and the second sound is the harmony interval; generating a target audio based on the first sound and the second sound, and the first sound and the second sound are presented as different harmony parts in the target audio. Thus, based on the triggering of the harmony control, different harmony sounds are realized according to the corresponding harmony parameters, which improves the human-computer interaction feeling, reduces the harmony addition cost, enriches the diversity of sound playback, improves the interestingness of sound playback, and adds a harmony effect based on the original first sound played, which improves the smoothness and naturalness of harmony addition.

Description

Technical Field

[0001] The present disclosure relates to the field of computer application technology, and in particular to a harmony processing method, device, equipment and medium. Background Art

[0002] With the development of computer technology, computer applications that enhance human-computer interaction are emerging in an endless stream. Among them, enhancing the sense of human-computer interaction in the dimension of sound is relatively common.

[0003] In the related technology, audio data is integrated into physical controls to realize human-computer interaction services in the sound dimension, that is, the corresponding audio data is integrated and set in the physical controls, and when it is sensed that the user triggers the corresponding physical control, the corresponding audio data is played to interact with the user's triggering operation.

[0004] However, the human-computer interaction service in the above-mentioned sound dimension not only requires the integrated setting of audio data and physical controls, which is costly, but also the amount of audio data is limited, resulting in the interactive sound playback being relatively simple and not very interesting. Summary of the invention

[0005] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a harmony processing method, device, equipment and medium to solve the problems of high human-computer interaction cost and poor human-computer interaction experience in the sound dimension in the prior art.

[0006] The embodiment of the present disclosure provides a harmony processing method, the method comprising: in response to a trigger operation on a target harmony control, obtaining a harmony interval corresponding to the target harmony control; performing voice-changing processing on the first sound originally input according to the harmony interval to obtain a second sound; wherein the interval between the first sound and the second sound is the harmony interval; generating a target audio according to the first sound and the second sound, in which the first sound and the second sound are presented as different harmony parts. The embodiment of the present disclosure also provides a harmony processing device, the device comprising: a first acquisition module, for obtaining a harmony interval corresponding to the target harmony control in response to a trigger operation on the target harmony control; a second acquisition module, for performing voice-changing processing on the first sound originally input according to the harmony interval to obtain a second sound; wherein the interval between the first sound and the second sound is the harmony interval; an audio generation module, for generating a target audio according to the first sound and the second sound, in which the first sound and the second sound are presented as different harmony parts.

[0007] An embodiment of the present disclosure also provides an electronic device, which includes: a processor; a memory for storing executable instructions executable by the processor; the processor is configured to read the executable instructions from the memory and execute the instructions to implement the harmony processing method provided by the embodiment of the present disclosure.

[0008] An embodiment of the present disclosure also provides a computer-readable storage medium, which stores a computer program for executing the harmony processing method provided by the embodiment of the present disclosure.

[0009] The technical solution provided by the embodiment of the present disclosure has the following advantages compared with the prior art: The harmony processing solution provided by the embodiment of the present disclosure, in response to a trigger operation on a target harmony control, obtains a harmony interval corresponding to the target harmony control, and performs pitch transformation processing on the original input first sound according to the harmony interval to obtain a second sound, wherein the interval between the first sound and the second sound is the harmony interval. Furthermore, a target audio is generated based on the first sound and the second sound, and the first sound and the second sound are presented as different harmony parts in the target audio. Thus, based on the trigger of the harmony control, different harmony sounds are realized according to the corresponding harmony intervals, which improves the human-computer interaction experience, reduces the cost of adding harmony, enriches the diversity of sound playback, improves the interest of sound playback, and adds the harmony effect based on the original played first sound, which improves the smoothness and naturalness of adding harmony. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In combination with the accompanying drawings and referring to the following specific embodiments, the above and other features, advantages and aspects of the embodiments of the present disclosure will become more obvious. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the original elements and elements are not necessarily drawn to scale.

[0011] Figure 1 It is a schematic flowchart of a harmony processing method provided by an embodiment of the present disclosure;

[0012] Figure 2 It is a schematic diagram of a harmony control display interface provided by an embodiment of the present disclosure;

[0013] Figure 3 It is a schematic diagram of a harmony processing scenario provided by an embodiment of the present disclosure;

[0014] Figure 4 It is another schematic diagram of a harmony processing scenario provided by an embodiment of the present disclosure;

[0015] Figure 5 It is a schematic diagram of a harmony processing logic node provided by an embodiment of the present disclosure;

[0016] Figure 6 Schematic flowchart of another harmony processing method provided by an embodiment of the present disclosure;

[0017] FIG. 7(a) is a schematic diagram of another harmony control display interface provided by an embodiment of the present disclosure;

[0018] FIG. 7(b) is a schematic diagram of another harmony control display interface provided by an embodiment of the present disclosure;

[0019] FIG. 8(a) is a schematic diagram of another harmony control display interface provided by an embodiment of the present disclosure;

[0020] FIG. 8(b) is a schematic diagram corresponding to musical notes and numbered musical notations provided by an embodiment of the present disclosure;

[0021] Figure 9 Schematic flowchart of another harmony processing method provided by an embodiment of the present disclosure;

[0022] Figure 10 Schematic diagram of another harmony processing logic node provided by an embodiment of the present disclosure;

[0023] Figure 11 Schematic flowchart of another harmony processing method provided by an embodiment of the present disclosure;

[0024] Figure 12 Schematic diagram of another harmony processing scenario provided by an embodiment of the present disclosure;

[0025] Figure 13 Schematic diagram of another harmony processing logic node provided by an embodiment of the present disclosure;

[0026] Figure 14 Schematic flowchart of another harmony processing method provided by an embodiment of the present disclosure;

[0027] Figure 15 Schematic diagram of another harmony processing logic node provided by an embodiment of the present disclosure;

[0028] Figure 16 Schematic diagram of the structure of a harmony processing device provided by an embodiment of the present disclosure;

[0029] Figure 17 Schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Detailed implementation manners

[0030] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0031] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0032] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0033] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0034] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0035] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0036] In order to solve the above problems, an embodiment of the present disclosure provides a harmony processing method, in which a flexible and varied harmony adding effect is achieved, and the harmony is added by fusion with the original input sound (such as human voice), so that the fused sound sounds more natural and more interesting.

[0037] The method is introduced below in conjunction with specific embodiments.

[0038] Figure 1 The present invention provides a flow chart of a method for processing harmony sounds, which can be performed by a device for processing harmony sounds, wherein the device can be implemented by software and / or hardware, and can generally be integrated into an electronic device.Figure 1 As shown, the method includes:

[0039] Step 101, in response to a trigger operation on a target harmony control, obtain a harmony interval corresponding to the target harmony control.

[0040] In this embodiment, the target harmony control can be a virtual control or a physical control, which is not limited herein. Additionally, a harmony interval refers to the mutual relationship of pitch levels of pitch classes. The number of harmony intervals can be one group or multiple groups. Each group of harmony intervals can be one or multiple. When there are multiple groups of harmony intervals, multiplexed corresponding harmony sounds can be achieved. The harmony interval can include a scale change value, a semitone number change value, a note change value, or chord audio data, etc.

[0041] In an embodiment of the present disclosure, multiple harmony controls can be preset, and the harmony intervals corresponding to each harmony control are different. Thus, different harmony intervals can be achieved based on the triggering of different harmony controls, so as to further achieve diverse harmony effects according to different harmony intervals.

[0042] In some possible embodiments, as Figure 2 shown, when there are 9 virtual controls as harmony controls, the 9 virtual controls can be directly displayed on a relevant interface. Among them, indication information of different harmony effects (such as Dm, Em, Am, etc. in the figure) can be displayed on different harmony controls, so that users can trigger the corresponding target harmony control according to the indication information. In this embodiment, the indication information can be, in addition to the above chord English letters, also style description information such as "bright", "dim", "tense", etc., or color filling information of the virtual control, etc.

[0043] Step 102, perform pitch transformation processing on the originally input first sound according to the harmony interval to obtain a second sound.

[0044] Wherein, the interval between the first sound and the second sound is the harmony interval.

[0045] The originally output first sound in this embodiment can be music playback data, voice input human voice data, etc. in a relevant application, which is not limited herein. The interval between the first sound and the second sound is the harmony interval, thus achieving the sound effect of a chord.

[0046] In this embodiment, the originally input first sound is processed according to the harmony interval. Since the harmony interval enables the user to achieve a harmony effect, the second sound that forms a harmony interval with the first sound can be obtained based on the harmony interval.

[0047] For example, when the harmonic interval includes multiple chromatic number change values, the chromatic number of the first sound can be adjusted in multiple ways to obtain the second sound of the corresponding other harmonic part, so as to realize the acquisition of the second sound that forms a harmonic interval with the first sound.

[0048] It should be noted that in some possible embodiments, the second sound can be obtained through the part. In this embodiment, the object of changing the harmonic part is the first sound. Therefore, obtaining the second sound based on the original input first sound, rather than adding a new harmonic audio that has no relation to the first sound, will improve the correlation between the second sound and the first sound, and further ensure the naturalness and smoothness of the fused sound.

[0049] For example, as Figure 3 shown, when the part trend of the corresponding sound is represented by a curve, if the harmonic intervals corresponding to the notes of each first sound are the same, when the part trend of the first sound is the upper curve in the figure, the part trend of the second sound after processing based on the first sound is related to the part trend of the first sound. Thus, the effect that the second sound comes from the original sound source of the first sound is realized. It should be noted that in actual applications, the part trend of the second sound does not have a strict linear correspondence with the part trend of the first sound. The figure is only a possible example and cannot be regarded as a limitation to the embodiments of the present disclosure.

[0050] Step 103, generate a target audio according to the first sound and the second sound, and the first sound and the second sound are presented as different harmonic parts in the target audio.

[0051] In this embodiment, the first sound and the second sound are mixed, thus enriching the part of the first sound, etc., realizing the playback effect of harmony. And since the second sound is obtained by processing the first sound, there is no need to separately record and store the second sound, etc., reducing the cost of adding harmony. For example, in an audio playback application, as Figure 4 shown, the corresponding game character may be singing, and what the user hears is the original input sound of the song "Twinkle, Twinkle, Little Star", for example, it can be recorded in advance by a certain user, etc. After the harmony processing of this embodiment, not only the original input sound of the song "Twinkle, Twinkle, Little Star" is simply played. In fact, what the user hears is the sound with the harmony effect of the song "Twinkle, Twinkle, Little Star".

[0052] To facilitate the understanding of the harmony processing method of the embodiments of the present disclosure by those skilled in the art, the following describes the execution logic of the harmony processing of the embodiments of the present disclosure in the form of functional module nodes:

[0053] In this embodiment, as Figure 5As shown, the original input's first sound is obtained through the StreamSourceNode node. Based on this first sound, the MDSPNode node processes the original input's first sound. Here, the processing of the original input's first sound by the MDSPNode node is determined according to the harmony interval corresponding to the triggered target harmony control. In this embodiment, each MDSPNode node corresponds to a group of harmony intervals. In the figure, there are two groups of harmony intervals. Further, the two transformed voices after processing the two groups of harmony intervals are mixed by the MixerNode mixing node to obtain the second sound. The second sound is input to the corresponding GainNode volume control node. When the value of the GainNode volume control node is 1, the second sound is output. When the value of the GainNode volume control node is 0, the second sound is not output. The value of the GainNode volume control node is related to whether the current user triggers the target harmony control. If the current user triggers the target harmony control, the value of the corresponding GainNode volume control node is 1. Otherwise, the value of the GainNode volume control node is 0.

[0054] Further, if the GainNode volume control node outputs the second sound, the first sound and the second sound are mixed by the SinkNode playback node for playback.

[0055] In summary, the harmony processing method of the present disclosure embodiment responds to the triggering operation of the target harmony control, obtains the harmony interval corresponding to the target harmony control, and then performs voice transformation processing on the original input's first sound according to the harmony interval to obtain the second sound. Among them, the interval between the first sound and the second sound is the harmony interval. The target audio is generated according to the first sound and the second sound, and the first sound and the second sound are presented as different harmony parts in the target audio. Thus, based on the triggering of the harmony control, different harmony sounds are realized according to the corresponding harmony interval, which improves the human-computer interaction feeling, reduces the harmony addition cost, enriches the diversity of sound playback, improves the interestingness of sound playback, and adds the harmony effect based on the original played first sound, which improves the smoothness and naturalness of harmony addition.

[0056] In actual execution, since the harmony interval will directly determine the harmony effect, the determination of the harmony interval is crucial.

[0057] It should be noted that in different application scenarios, the method of obtaining the harmony interval corresponding to the target harmony control in response to the triggering operation of the target harmony control is different. The examples are as follows:

[0058] In one embodiment of the present disclosure, the harmonic intervals and the harmonic controls are also stored in the form of a corresponding relationship. Specifically, when storing, the control identifier corresponding to the harmonic control can be obtained. The control identifier can be any information such as control color, control indication information, control position, etc. that can uniquely determine the corresponding harmonic control. In this embodiment, in response to a triggering operation on the target harmonic control, the preset corresponding relationship between the control identifier and the harmonic interval is queried, and the harmonic interval corresponding to the control identifier of the target harmonic control is obtained.

[0059] In another embodiment of the present disclosure, as Figure 6 shown, in response to a triggering operation on the target harmonic control, obtaining the harmonic interval corresponding to the target harmonic control includes:

[0060] Step 601, in response to a triggering operation on the target harmonic control, determine the target chord type interval set corresponding to the target harmonic control.

[0061] In this embodiment, the harmonic control corresponds to a chord type interval set. As shown in FIG. 8(a), the chord types include major triads with bright colors composed of numbered musical notations "135", "461", "572", etc., minor triads with dull colors composed of numbered musical notations "246", "357", "613", etc., diminished triads with tense colors composed of numbered musical notations "724", etc. The chord type interval set is a list composed of the number of notes corresponding to the numbered musical notations corresponding to the above chord types. It should be noted that the numbered musical notations and the corresponding chord types in the figure are only a possible example proposed for the convenience of understanding the embodiments of the present disclosure and should not be regarded as a limitation to the embodiments of the present disclosure.

[0062] Thus, in some possible embodiments, in response to a triggering operation on the target harmonic control, the preset corresponding relationship between the control identifier and the chord type can be queried to determine the target chord type interval set corresponding to the control identifier of the target harmonic control.

[0063] In this embodiment, as shown in FIG. 7(a), only one sound control corresponding to the same type of chord type can be displayed, and each sound control corresponds to one chord type.

[0064] In some other possible embodiments, to enhance the intuitiveness of selection, in response to a triggering operation on a target harmony control, a chord type control associated with the target harmony control may be activated to send corresponding prompt information. The prompt information may be any form of information indicating the chord type. For example, as shown in FIG. 7(b), activating the chord type control associated with the target harmony control to send corresponding prompt information is the English letter of the corresponding chord. For example, sending the corresponding prompt information to a relevant server. Furthermore, according to the prompt information, a chord type interval set corresponding to the control identifier of the chord type control is determined as the target chord type interval set.

[0065] In this embodiment, the harmony control corresponds to a chord type interval set. Among them, as shown in FIG. 8(a), different harmony controls may correspond to the same chord type interval set. For example, the major chord type interval set in FIG. 8(a) corresponds to the three harmony controls in the first row.

[0066] Step 602, query the preset correspondence between the control identifier and each note in the target chord type interval set, and obtain the target note corresponding to the control identifier of the target harmony control.

[0067] Continuing to refer to FIG. 8(a), although different harmony controls may correspond to the same chord type interval set, different harmonies correspond to different numbered musical notations of the chord type interval set, and thus different numbered musical notations correspond to different notes. Therefore, in this embodiment, the preset correspondence between the control identifier and each note in the target chord type interval set is queried, and the target note corresponding to the control identifier of the target harmony control is obtained. The control identifier here may refer to the above embodiment and will not be elaborated here.

[0068] Among them, the notes in this embodiment are related to the number of tones (including the number of semitones and whole tones, etc.). The variation rule of the number of tones of the notes of the same chord type is the same. For example, referring to FIG. 8(b), the notes corresponding to the major chord numbered musical notation "135" are "0-4-7", and the number of tones of the notes corresponding to the major chord numbered musical notation "461" is "5-9-12". The semitone intervals are both 4 and 3. Thus, the same chord auditory effect is achieved based on the same variation of the number of tones.

[0069] Among them, if the same harmony control corresponds to multiple notes, then when determining the target note of the target harmony control subsequently, any one of the multiple notes corresponding to the multiple harmony controls of the same type of chord type can be randomly selected based on a random algorithm as the target note of the triggered target harmony control.

[0070] Step 603, query the target chord type interval set and obtain the harmony note corresponding to the target note.

[0071] In this embodiment, query the target chord type interval set to obtain the harmonic notes corresponding to the target note. In this embodiment, the corresponding harmonic notes can be obtained by querying based on a preset mapping relationship. Since a chord is composed of multiple harmonic notes, therefore, other harmonic notes with the same chord type under the chord type corresponding to the target harmony control can also be determined as the harmonic notes corresponding to the target note.

[0072] Step 604, determine the harmony interval corresponding to the target harmony control according to the note change between the target note and the harmonic note.

[0073] In this embodiment, in order to achieve the chord effect, as analyzed above, it is achieved based on the change in the number of tones corresponding to different notes. Therefore, for the harmony to achieve the chord effect, in this embodiment, according to the note change between the target note and the harmonic note, that is, the change in the number of tones, the harmony interval corresponding to the target harmony control is determined.

[0074] For example, when the chord type of the selected target harmony control is a minor triad, the corresponding numbered musical notation of the target harmony control is in root position, and the singer sings "re", then the minor triad is composed of the numbered musical notation "246". Therefore, determine that the note changes corresponding to "46" and "2" are the harmony intervals corresponding to the target harmony control. In this example, since the number of half tones corresponding to "2" is "2", the number of half tones corresponding to "4" is "5", and the number of half tones corresponding to "6" is "6", therefore, determine the corresponding note changes as "+3" and "+4". Thus, the number of tones of the note corresponding to the first sound can be changed by "+3" and "+4" to achieve the sound effect of adding a minor triad.

[0075] It should be noted that in this embodiment, for the same type of chord type, due to the different orders of the target notes corresponding to the triggered target harmony control in the chord type interval set, multiple different types of chord effects can be achieved. For example, for a major triad, if the note triggered is the first note of the major triad, the chord effect achieved is the chord effect composed of the two sounds obtained by changing the number of tones of the first sound by "+4" and "+7". If the note triggered is the second note of the major triad, the chord effect achieved is the chord effect composed of the two sounds obtained by changing the number of tones of the first sound by "-4" and "+3". If the note triggered is the third note of the major triad, the chord effect achieved is the chord effect composed of the two sounds obtained by changing the number of tones of the first sound by "-7" and "-3". Thus, the diversity of chord types is greatly expanded, enriching the fun of sound playback.

[0076] In still another embodiment of the present disclosure, it is also possible to identify the pitch type of the input first voice and query a preset database based on the pitch type to obtain the corresponding harmony interval.

[0077] In summary, the harmony processing method according to the embodiments of the present disclosure flexibly selects and determines the method for the harmony interval corresponding to the target harmony control according to the scene requirements. Based on different implementation methods of the harmony interval, the possibilities of the harmony interval are expanded. Thus, the diversity of harmony is expanded, and the interest of the played sound is enhanced. Moreover, when determining the harmony interval, it depends on the user's triggering operation on the target harmony control. Different target harmony controls triggered by the user result in different finally achieved harmony effects, enhancing the interest of human-computer interaction.

[0078] Based on the above embodiments, in order to further enhance the atmosphere of sound playback, other-dimensional processing can also be performed to assist in the mixed playback of the second sound, enhancing the interest of sound playback.

[0079] In an embodiment of the present disclosure, as Figure 9 shown, the method further includes:

[0080] Step 901, obtain the background sound of the current scene.

[0081] Among them, the background sound can be determined by querying the preset corresponding relationship according to the scene type. For example, when the scene type is a game scene, the background sound can be electronic music, etc. For example, it can be determined according to the content type of the first sound. When the first sound is the children's song "Twinkle, Twinkle, Little Star", the corresponding background sound can be queried from the preset database as the sound of the night sea breeze, etc.

[0082] Of course, in some possible implementation manners, the user can also manually select the corresponding background sound on the relevant interface.

[0083] Step 902, mix the first sound, the second sound, and the background sound for playback.

[0084] In this embodiment, mixing the first sound, the second sound, and the background sound for playback further enhances the music playback effect.

[0085] In terms of execution logic, as Figure 10 shown, at the function node as Figure 5 shown, a FileSourceNode node can also be included. The FileSourceNode node is connected to the SinkNode playback node and is used to transmit the confirmed background music.

[0086] In another embodiment of the present disclosure, as Figure 11 shown, the method further includes:

[0087] Step 1101, detect the volume of the first sound.

[0088] In this embodiment, if the first sound is related to the playback of an application, the volume of the corresponding first sound can be determined based on the length of the volume control bar of the application on the terminal device where the application is located, etc.

[0089] Step 1102, obtain the rendering parameters of the application scenario according to the volume of the first sound.

[0090] Among them, the rendering parameters include one or a combination of multiple of color, animation, pattern, text, etc. The application scenario is the application scenario corresponding to the first sound, such as a game scenario, a music playback scenario, etc.

[0091] Step 1103, control the application scenario according to the rendering parameters.

[0092] In an embodiment of the present disclosure, the renderable area in the application scenario can also be specified in advance and rendered in the render area according to the rendering parameters. Or, the target object to be rendered can be determined according to deep learning technology, and the corresponding target object can be rendered according to the rendering parameters.

[0093] In this embodiment, the display based on the rendering parameters reflects the change of the first volume, thereby further improving the sound playback effect. For example, Figure 12 as shown, when the application scenario is a cartoon character singing scenario, the rendering parameter is the parameter of the area size of the glowing area in the background area where the cartoon character is located. When the first sound is large, the background area corresponding to the cartoon character shows a larger glowing area. When the first sound is small, the background area corresponding to the cartoon character shows a smaller glowing area, etc.

[0094] For another example, when the application scenario is a cartoon character singing scenario, the rendering parameter is the control of the stage height where the cartoon character is located. When the first sound is large, the stage height corresponding to the cartoon character is higher. When the first sound is small, the stage height corresponding to the cartoon character is lower, etc.

[0095] Logically, as Figure 13 shown, at the function node as Figure 5 shown, it can also include a VolumeDetection node. The VolumeDetection node is connected to the StreamSourceNode node, used to detect the volume of the first sound, match the rendering parameters based on the first volume, and control the application scenario based on the rendering parameters.

[0096] In another embodiment of the present disclosure, as Figure 14 shown, the method further includes:

[0097] Step 1401, monitor the position of the character in the application scenario.

[0098] In this embodiment, the person in the application scenario can be a virtual object like the above-mentioned cartoon character, which is used to indicate the sound source of the first sound auditorily. The corresponding person position can be the position of the virtual person in the video screen, or the position on the stage, etc. Of course, in the case of a video shooting scenario, the person position can be the position of the actual shooting object in the real scene, which is the position from the camera of the terminal device, etc.

[0099] Step 1402: Adjust the sound source position of the second sound according to the person position.

[0100] It is easy to understand that different sound emission positions result in different sound playback effects. Therefore, in this embodiment, the sound source position of the second sound can also be adjusted according to the person position to form a stereo playback effect.

[0101] In some possible embodiments, the sound parameter distribution corresponding to different person positions (including audio speed, audio oscillation amplitude, sound volume size change, etc.) can be determined according to a preset database, and the volume of the second sound can be adjusted according to the sound parameter distribution to achieve the effect of adjusting the sound source position of the second volume auditorily.

[0102] In terms of execution logic, as Figure 15 shown, on the function node as Figure 5 shown, there can also be a SpatializerNode node, which is used to adjust the sound source position of the second sound for the second sound and input the adjusted second sound into the corresponding MixerNode mixing node to mix the first sound and the second sound to achieve a stereo playback effect.

[0103] In summary, the harmony processing method of the embodiments of the present disclosure adopts different methods to improve the sound playback effect after mixing the second sound and the first sound, and further enhances the interest of harmony playback.

[0104] To implement the above embodiments, the present disclosure also proposes a harmony processing device.

[0105] Figure 16 FIG. is a schematic structural diagram of a harmony processing device provided by an embodiment of the present disclosure. This device can be implemented by software and / or hardware and is generally integrated in an electronic device. As Figure 16 shown, the device includes: a first acquisition module 1610, a second acquisition module 1620, and an audio generation module 1630, where,

[0106] The first acquisition module 1610 is configured to acquire a harmony interval corresponding to a target harmony control in response to a trigger operation on the target harmony control;

[0107] A second acquisition module 1620, configured to perform voice conversion on a first voice of an original input according to a harmonic interval to obtain a second voice; wherein, the interval between the first voice and the second voice is a harmonic interval;

[0108] An audio generation module 1630, configured to generate a target audio according to the first voice and the second voice, and the first voice and the second voice are presented as different harmonic voices in the target audio.

[0109] The harmonic processing device provided by the embodiments of the present disclosure can execute the harmonic processing method provided by any embodiment of the present disclosure, and has corresponding functional modules and beneficial effects for executing the method.

[0110] To implement the above embodiments, the present disclosure also proposes a computer program product, including a computer program / instructions, which when executed by a processor, implement the harmonic processing method in the above embodiments

[0111] Figure 17 It is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure.

[0112] Specifically refer to the following Figure 17 , which shows a schematic structural diagram of an electronic device 1700 suitable for implementing the embodiments of the present disclosure. The electronic device 1700 in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 17 The electronic device shown is only an example, and should not impose any limitation on the functions and usage scope of the embodiments of the present disclosure.

[0113] As Figure 17 shown, the electronic device 1700 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 1701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1702 or a program loaded from a storage device 1708 into a random access memory (RAM) 1703. In the RAM 1703, various programs and data required for the operation of the electronic device 1700 are also stored. The processing device 1701, the ROM 1702, and the RAM 1703 are connected to each other through a bus 1704. An input / output (I / O) interface 1705 is also connected to the bus 1704.

[0114] Typically, the following devices can be connected to the I / O interface 1705: an input device 1706 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 1707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1708 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1709. The communication device 1709 can allow the electronic device 1700 to communicate with other devices wirelessly or wiredly to exchange data. Although Figure 17 the electronic device 1700 with various devices is shown, it should be understood that it is not required to implement or have all the shown devices. More or fewer devices can be alternatively implemented or had.

[0115] Specifically, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes program codes for performing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network through the communication device 1709, or installed from the storage device 1708, or installed from the ROM 1702. When the computer program is executed by the processing device 1701, the above functions defined in the harmony processing method of the embodiment of the present disclosure are executed.

[0116] It should be noted that the above-mentioned computer-readable medium in the present disclosure can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device. In the present disclosure, the computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, in which the computer-readable program code is carried. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, and this computer-readable signal medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0117] In some embodiments, the client and the server can communicate using any currently known or future-developed network protocol such as HTTP (HyperText Transfer Protocol), and can be interconnected with digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0118] The above-mentioned computer-readable medium can be included in the above-mentioned electronic device; or it can exist separately without being assembled into the electronic device.

[0119] The above computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: in response to a trigger operation on a target harmony control, obtain a harmony interval corresponding to the target harmony control, perform pitch-shifting processing on a first sound of the original input according to the harmony interval to obtain a second sound, where the interval between the first sound and the second sound is the harmony interval. Furthermore, generate a target audio based on the first sound and the second sound, and the first sound and the second sound are presented as different harmony parts in the target audio. Thus, based on the trigger of the harmony control, different harmony sounds are realized according to the corresponding harmony intervals, enhancing the human-computer interaction experience, reducing the cost of adding harmony, enriching the diversity of sound playback, increasing the fun of sound playback, and adding harmony effects based on the original first sound being played, enhancing the smoothness and naturalness of harmony addition.

[0120] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages or combinations thereof. The programming languages include, but are not limited to, object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may execute entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0121] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that, in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system for performing the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0122] The units involved in the embodiments of the present disclosure can be implemented in software or in hardware. In some cases, the name of the unit does not constitute a limitation on the unit itself.

[0123] The functions described above herein can be performed, at least in part, by one or more hardware logic components. By way of example, and without limitation, exemplary types of hardware logic components that may be used include: Field Programmable Gate Arrays (FPGA), Application Specific Integrated Circuits (ASIC), Application Specific Standard Products (ASSP), System on a Chip (SOC), Complex Programmable Logic Devices (CPLD), and the like.

[0124] In the context of the present disclosure, a machine-readable medium may be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a Read-Only Memory (ROM), an Erasable Programmable Read-Only Memory (EPROM or Flash memory), an optical fiber, a portable Compact Disc Read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0125] According to one or more embodiments of the present disclosure, the present disclosure provides a harmony processing method, including:

[0126] In response to a trigger operation on a target harmony control, obtaining a harmony interval corresponding to the target harmony control;

[0127] Performing pitch shifting processing on a first sound of an original input according to the harmony interval to obtain a second sound; wherein, the interval between the first sound and the second sound is the harmony interval;

[0128] Generating a target audio according to the first sound and the second sound, wherein the first sound and the second sound are presented as different harmony parts in the target audio.

[0129] According to one or more embodiments of the present disclosure, in the harmony processing method provided by the present disclosure, the step of in response to a trigger operation on a target harmony control, obtaining a harmony interval corresponding to the target harmony control includes:

[0130] In response to a triggering operation on a target harmony control, query the preset correspondence between control identifiers and harmony intervals, and obtain the harmony interval corresponding to the control identifier of the target harmony control.

[0131] According to one or more embodiments of the present disclosure, in the harmony processing method provided by the present disclosure, the obtaining the harmony interval corresponding to the target harmony control in response to a triggering operation on the target harmony control includes:

[0132] In response to a triggering operation on a target harmony control, determine a target chord type interval set corresponding to the target harmony control;

[0133] Query the preset correspondence between control identifiers and each note in the target chord type interval set, and obtain the target note corresponding to the control identifier of the target harmony control;

[0134] Query the target chord type interval set to obtain the harmony note corresponding to the target note;

[0135] Determine the harmony interval corresponding to the target harmony control according to the note change between the target note and the harmony note.

[0136] According to one or more embodiments of the present disclosure, in the harmony processing method provided by the present disclosure, the determining a target chord type interval set corresponding to the target harmony control in response to a triggering operation on the target harmony control includes:

[0137] In response to a triggering operation on a target harmony control, query the preset correspondence between control identifiers and chord types, and determine the target chord type interval set corresponding to the control identifier of the target harmony control.

[0138] According to one or more embodiments of the present disclosure, in the harmony processing method provided by the present disclosure, the determining a target chord type interval set corresponding to the target harmony control in response to a triggering operation on the target harmony control includes:

[0139] In response to a triggering operation on a target harmony control, start the chord type control associated with the target harmony control to send a corresponding prompt message;

[0140] Determine that the chord type interval set corresponding to the control identifier of the chord type control is the target chord type interval set according to the prompt message.

[0141] According to one or more embodiments of the present disclosure, the harmony processing method provided by the present disclosure further includes:

[0142] Obtain the background sound of the current scene;

[0143] Play by mixing the first sound, the second sound, and the background sound.

[0144] According to one or more embodiments of the present disclosure, in the harmony processing method provided by the present disclosure, it further includes:

[0145] Detect the volume of the first sound;

[0146] Obtain the rendering parameters of the application scenario according to the volume of the first sound;

[0147] Control the application scenario according to the rendering parameters.

[0148] According to one or more embodiments of the present disclosure, in the harmony processing method provided by the present disclosure, it further includes:

[0149] Monitor the position of the person in the application scenario;

[0150] Adjust the sound source position of the second sound according to the position of the person.

[0151] According to one or more embodiments of the present disclosure, the present disclosure provides a harmony processing device, including:

[0152] A first acquisition module, configured to obtain the harmony interval corresponding to the target harmony control in response to a trigger operation on the target harmony control;

[0153] A second acquisition module, configured to perform pitch-shifting processing on the originally input first sound according to the harmony interval to obtain a second sound; wherein, the interval between the first sound and the second sound is the harmony interval;

[0154] An audio generation module, configured to generate a target audio according to the first sound and the second sound, and the first sound and the second sound are presented as different harmony parts in the target audio.

[0155] According to one or more embodiments of the present disclosure, in the harmony processing device provided by the present disclosure, the first acquisition module is specifically configured to:

[0156] In response to a trigger operation on the target harmony control, query the corresponding relationship between the preset control identifier and the harmony interval, and obtain the harmony interval corresponding to the control identifier of the target harmony control.

[0157] According to one or more embodiments of the present disclosure, in the harmony processing device provided by the present disclosure, the first acquisition module is specifically configured to:

[0158] In response to a trigger operation on the target harmony control, determine the target chord type interval set corresponding to the target harmony control;

[0159] Query the correspondence between the preset control identifier and each note in the target chord type interval set, and obtain the target note corresponding to the control identifier of the target harmony control;

[0160] Query the target chord type interval set, and obtain the harmony note corresponding to the target note;

[0161] Determine the harmony interval corresponding to the target harmony control according to the note change between the target note and the harmony note.

[0162] According to one or more embodiments of the present disclosure, in the harmony processing device provided by the present disclosure, the first acquisition module is specifically configured to:

[0163] In response to a trigger operation on the target harmony control, query the correspondence between the preset control identifier and the chord type, and determine the target chord type interval set corresponding to the control identifier of the target harmony control.

[0164] According to one or more embodiments of the present disclosure, in the harmony processing device provided by the present disclosure, the first acquisition module is specifically configured to:

[0165] In response to a trigger operation on the target harmony control, start the chord type control associated with the target harmony control to send a corresponding prompt message;

[0166] Determine that the chord type interval set corresponding to the control identifier of the chord type control is the target chord type interval set according to the prompt message.

[0167] According to one or more embodiments of the present disclosure, the harmony processing device provided by the present disclosure further includes:

[0168] A third acquisition module for acquiring the background sound of the current scene;

[0169] The audio generation module is further configured to mix the first sound, the second sound, and the background sound for playback.

[0170] According to one or more embodiments of the present disclosure, the harmony processing device provided by the present disclosure further includes:

[0171] A detection module for detecting the volume of the first sound;

[0172] A fourth acquisition module for acquiring the rendering parameters of the application scenario according to the volume of the first sound;

[0173] A control module for controlling the application scenario according to the rendering parameters.

[0174] According to one or more embodiments of the present disclosure, the harmony processing device provided by the present disclosure further includes:

[0175] A monitoring module for monitoring the positions of people in an application scenario;

[0176] An adjustment module for adjusting the sound source position of the second sound according to the positions of the people.

[0177] According to one or more embodiments of the present disclosure, the present disclosure provides an electronic device, including:

[0178] A processor;

[0179] A memory for storing executable instructions of the processor;

[0180] The processor is configured to read the executable instructions from the memory and execute the instructions to implement any one of the harmony processing methods provided by the present disclosure.

[0181] According to one or more embodiments of the present disclosure, the present disclosure provides a computer-readable storage medium storing a computer program for executing any one of the harmony processing methods provided by the present disclosure.

[0182] The above description is only a preferred embodiment of the present disclosure and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosure concept. For example, the technical solutions formed by mutually replacing the above features with the (but not limited to) technical features having similar functions disclosed in the present disclosure.

[0183] In addition, although the operations are depicted in a specific order, this should not be construed as requiring the operations to be performed in the specific order shown or in sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, although a number of specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present disclosure. Certain features described in the context of separate embodiments may also be implemented combinatorially in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented separately or in any suitable sub-combination in multiple embodiments.

[0184] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms for implementing the claims.

Claims

1. A harmony processing method, characterized in that, Including: In response to a triggering operation on a target harmony control, obtaining a target chord type interval set corresponding to the target harmony control; Obtaining a target note corresponding to the control identifier of the target harmony control and a harmony note corresponding to the target note according to the target chord type interval set, and determining a harmony interval according to the note change between the harmony note and the target note; Performing pitch-shifting processing on the originally input first sound according to the harmony interval to obtain a second sound; wherein, the interval between the first sound and the second sound is the harmony interval; Generating a target audio according to the first sound and the second sound, and presenting the first sound and the second sound as different harmony parts in the target audio.

2. The method according to claim 1, characterized in that, The obtaining a target note corresponding to the control identifier of the target harmony control and a harmony note corresponding to the target note according to the target chord type interval set includes: Querying a preset correspondence between control identifiers and each note in the target chord type interval set to obtain a target note corresponding to the control identifier of the target harmony control; Querying the target chord type interval set to obtain a harmony note corresponding to the target note.

3. The method according to claim 2, characterized in that, The responding to a triggering operation on a target harmony control and obtaining a target chord type interval set corresponding to the target harmony control includes: In response to a triggering operation on a target harmony control, querying a preset correspondence between control identifiers and chord types to determine a target chord type interval set corresponding to the control identifier of the target harmony control.

4. The method according to claim 1, characterized in that, The responding to a triggering operation on a target harmony control and determining a target chord type interval set corresponding to the target harmony control includes: In response to a triggering operation on a target harmony control, starting a chord type control associated with the target harmony control to send a corresponding prompt message; Determining, according to the prompt message, a chord type interval set corresponding to the control identifier of the chord type control as the target chord type interval set.

5. The method according to claim 1, characterized in that, Further including: Obtaining the background sound of the current scene; Mixing the first sound, the second sound, and the background sound for playback.

6. The method according to any one of claims 1-5, characterized in that, Further including: Detecting the volume of the first sound; Obtaining rendering parameters of the application scenario according to the volume of the first sound; Controlling the application scenario according to the rendering parameters.

7. The method according to any one of claims 1-5, characterized in that, Further including: Monitoring the position of a person in the application scenario; Adjusting the sound source position of the second sound according to the position of the person.

8. A harmony processing device, characterized in that, Including: A first obtaining module, configured to, in response to a triggering operation on a target harmony control, obtain a target chord type interval set corresponding to the target harmony control, obtain a target note corresponding to the control identifier of the target harmony control and a harmony note corresponding to the target note according to the target chord type interval set, and determine a harmony interval according to the note change between the harmony note and the target note; A second obtaining module, configured to perform pitch-shifting processing on the originally input first sound according to the harmony interval to obtain a second sound; wherein, the interval between the first sound and the second sound is the harmony interval; An audio generation module, configured to generate a target audio according to the first sound and the second sound, wherein the first sound and the second sound are presented as different harmony voices in the target audio.

9. An electronic device, characterized in that, The electronic device includes: a processor; a memory for storing executable instructions of the processor; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the harmony processing method according to any one of claims 1-7 above.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and the computer program is used to execute the harmony processing method according to any one of claims 1-7 above.

Images