Music playback method, device, equipment, and storage medium based on user interaction

Abstract

A music playback method, device, equipment, and storage medium based on user interaction, which relates to the technical field of music videos, includes the steps of collecting a target video using a video capture device, determining an audio processing method corresponding to the target action when a target object in the target video is identified to perform a target action, and playing music according to the audio processing method. By collecting a target video using a video capture device and determining an audio processing method corresponding to the target action when a target object in the target video is identified to perform a target action, and playing music according to the audio processing method, users can freely create personalized music based on their own actions using the video, which eliminates restrictions on music creation, increases the interest of music creation, and improves the user experience.

Description

[Technical Field]

[0001] This application claims priority from Chinese Patent Application No. 202011163846.9 filed on October 27, 2020, the entire contents of which are incorporated herein by reference.

[0002] The present disclosure relates to a music playback method, device, apparatus, and storage medium based on user interaction. [Background technology]

[0003] Nowadays, multimedia interaction platforms have become an important part of people's lives, and users can record videos on these platforms and create different video contents according to different music.

[0004] However, these known platforms only allow users to create video content based on music selected by the user when recording a video, and all users can only create videos based on music provided by the system or music selected locally by the user or downloaded from the Internet, which does not meet the needs of music lovers to freely edit music according to their own preferences and create personalized music that belongs to them.

[0005] As can be seen, when users select and create music through a multimedia interaction platform, the music they select is all known, resulting in insufficient personalization of the music created by the users and a poor user experience. Summary of the Invention [Means for solving the problem]

[0006] The purpose of the present disclosure is to at least solve one of the above technical deficiencies, in particular the technical deficiency that when a user selects and creates music through a multimedia interaction platform, the selected music is all known, resulting in insufficient personalization of the music created by the user and a poor user experience.

[0007] In a first aspect, there is provided a method for playing music based on user interaction, the method comprising: collecting a target video with a video capture device; determining a corresponding audio processing manner of the target action when identifying that the target object exhibits the target action in the target video; and playing music according to the audio processing scheme.

[0008] In a second aspect, there is provided a user-operated music playback device, the device comprising: a target video acquisition module for acquiring the target video by a video capture device; a target action identification module for determining a corresponding audio processing manner of the target action when identifying that the target object exhibits a target action in the target video; and a music playback module for playing music according to the audio processing method.

[0009] In a third aspect, there is provided an electronic device, the electronic device comprising: one or more processors; Memory and and one or more application programs stored in the memory and configured to be executed by the one or more processors and configured to perform the user interaction based music playback method.

[0010] In a fourth aspect, there is provided a storage medium, wherein the storage medium stores at least one instruction, at least one segment of a program, a code set or an instruction set, and the at least one instruction, the at least one segment of a program, the code set or the instruction set is uploaded and executed by a processor to realize the music playback method based on user interaction.

[0011] In an embodiment of the present disclosure, a target video is collected by a video capture device, and when it is identified that a target object in the target video performs a target action, a corresponding audio processing method is determined based on the target action, and music is played according to the audio processing method, so that the user can freely create personalized music based on their own actions through the video, which does not restrict music creation, but also increases the interest of music creation and improves the user experience.

[0012] In order to more clearly describe the technical solutions in the embodiments of the present disclosure, the following briefly describes the drawings that need to be used to describe the embodiments of the present disclosure. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a flowchart of a music playing method based on user interaction according to an embodiment of the present disclosure. [Figure 2] FIG. 2 is a schematic diagram of a musical material identifier display according to an embodiment of the present disclosure. [Figure 3] FIG. 3 is a flowchart of a method for selecting an identifier for music material according to an embodiment of the present disclosure. [Figure 4] FIG. 4 is a schematic diagram of a selection of music materials according to an embodiment of the present disclosure. [Figure 5] FIG. 5 is a schematic diagram illustrating an interactive operation acquisition according to an embodiment of the present disclosure. [Figure 6] FIG. 6 is a structural schematic diagram of a music playback device based on user interaction according to an embodiment of the present disclosure. [Figure 7]FIG. 7 is a structural schematic diagram of a device according to an embodiment of the present disclosure.

[0014] These and other features, advantages, and aspects of each embodiment of the present disclosure will become more apparent by reference to the following specific embodiments in conjunction with the drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It is to be understood that the drawings are illustrative and that objects and elements are not necessarily drawn to scale. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, embodiments of the present disclosure will be described in more detail with reference to the drawings. Although the drawings show several embodiments of the present disclosure, it should be understood that the present disclosure may be realized in various forms and should not be understood as being limited to the embodiments described herein, but rather, these embodiments are provided to enable a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are merely illustrative and do not limit the scope of protection of the present disclosure.

[0016] It will be understood that the steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel, and that method embodiments may include additional steps and / or omit performing steps that are shown, and the scope of the present disclosure is not limited in this respect.

[0017] As used herein, the term "comprises" and variations thereof are intended to be open-ended, i.e., "including, but not limited to." The term "based on" means "based at least in part on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one other embodiment," and the term "some embodiments" means "at least some embodiments." Relevant definitions of other terms are provided in the description below.

[0018] It should be noted that concepts such as "first" and "second" referred to in this disclosure are merely intended to distinguish between devices, modules, or units, and do not limit these devices, modules, or units to being necessarily different devices, modules, or units, nor do they limit the order or interdependence of functions performed by these devices, modules, or units.

[0019] It should be noted that the modifications "one" and "multiple" referred to in this disclosure are intended to be exemplary rather than limiting, and should be understood as "one or more" unless the context clearly indicates otherwise, as would be understood by one of ordinary skill in the art.

[0020] The names of interactive messages or information between devices in the embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0021] The music playback method, device, electrical device, and storage medium based on user interaction according to the present disclosure aim to solve the above technical problems of the prior art.

[0022]

[0023] The following specific examples will be used to describe in detail the technical solutions of the present disclosure and how the technical solutions of the present disclosure solve the above technical problems. Some of the following specific examples can be combined with each other, and detailed descriptions of the same or similar concepts or processes may be omitted in some examples.

[0024] The following examples will be described with reference to the drawings.

[0023] An embodiment of the present disclosure provides a music playing method based on user interaction, and as shown in FIG. 1 , the method includes: Step S101: collecting a target video by a video capture device; If it is identified that the target object in the target video exhibits a target action, determining an audio processing method corresponding to the target action in step S102; and step S103 of playing music according to the audio processing method.

[0024] The music playback method based on user interaction according to an embodiment of the present disclosure is applied to a terminal device, and may be an APP (Application) in the terminal device, or a function in an APP. The user terminal may be provided with a video capture device, and optionally a front camera. The user can trigger a music playback trigger operation of the music playback method through the terminal device. The terminal device captures a target video through the video capture device based on the music playback trigger operation. If the target object in the target video exhibits a target action, the terminal device determines a corresponding audio processing method based on the target action and plays music according to the audio processing method, allowing the user to freely create personalized music based on their own operations. This eliminates restrictions on music creation, increases the interest of music creation, and improves the user experience.

[0025] In an embodiment of the present disclosure, the music playback trigger operation refers to a user opening an APP for implementing the user interaction-based music playback method of the present disclosure, or an operation of activating a function in the APP implemented based on the method, where the function can implement the user interaction-based music playback method of the present disclosure. In an embodiment of the present disclosure, the target object may be a part of a person's body in the target video, such as a hand, foot, mouth, etc., and the target action may be a hand or foot movement or opening and closing of the mouth, etc. In an embodiment of the present disclosure, a corresponding audio processing manner can be determined based on the target action shown by the target object, and music can be played according to the audio processing manner.

[0026]

[0013] To facilitate the description of the embodiments of the present disclosure, a specific embodiment will be taken as an example. The user interaction-based music playback method according to the embodiments of the present disclosure can be implemented as a music editing function in an APP. The user's music generation trigger operation can be an operation to open the function in the APP, such as clicking a virtual button corresponding to the function in the APP interface, or activating the function through voice, gesture, or other operations. In response to the music generation trigger operation, the terminal device collects a target video using a video capture device, optionally displays the target video on a display interface of the terminal device, and identifies content in the target video. In one embodiment of the present disclosure, if it is identified that a target object is present in the target video and that the target object exhibits a target action, an audio processing manner corresponding to the target action is determined. Optionally, the target object can be a part of a human body in the target video, such as a hand, foot, mouth, etc., and the target action can be a hand or foot movement or opening and closing the mouth, etc. As one embodiment of the present disclosure, the target object may exhibit a target action by a human hand in the target video touching a certain part of the playback interface in the video; optionally, when determining a corresponding audio processing method of the target action, an audio processing method related to the part can be determined based on the part corresponding to the target; optionally, audio processing methods corresponding to different parts are different, and after determining the corresponding audio processing method, music is played according to the audio processing method.

[0027] In an embodiment of the present disclosure, a target video is captured by a video capture device, and when it is identified that a target object in the target video exhibits a target action, a corresponding audio processing method is determined based on the target action, and music is played according to the audio processing method, so that the user can freely create personalized music based on their own operations, which does not restrict music creation, but also increases the interest of music creation and improves the user experience.

[0028] The embodiments of the present disclosure provide one possible implementation, in which the audio processing method specifically adds a target soundtrack corresponding to the target action to the music and plays it.

[0029] In the embodiment of the present disclosure, when playing music according to the soundtrack processing method, a target soundtrack corresponding to the target action may be added to the music and played.

[0030] In the embodiments of the present disclosure, different target actions can correspond to different audio processing schemes, and different audio processing schemes can play different soundtracks correspondingly. Optionally, when playing music according to an audio processing scheme, the soundtrack corresponding to the audio processing scheme can be added to the music and played. In one embodiment of the present disclosure, the soundtrack can be a predetermined piece of music, or music played on a predetermined instrument, or optionally a portion of the predetermined music played on a certain instrument. When determining a corresponding audio processing scheme based on a target action, the corresponding soundtrack can be determined based on the target action and the soundtrack can be added to the music and played. In one embodiment of the present disclosure, if the soundtrack determined based on the target action is a piece of music played on a piano, the music played on the piano can be added to the music and played when playing music according to an audio processing scheme.

[0031] The embodiment of the present disclosure determines a corresponding audio processing method based on a target action, and determines a corresponding soundtrack. When playing music according to the audio processing method, the corresponding soundtrack is added to the music and played. The user can add different soundtracks to the music according to the action, which increases the possibility of music creation.

[0032] The embodiments of the present disclosure provide one possible implementation form, in which the target action is specifically a touch action of the target object on a target position in the playback interface of the target video, and the target position corresponds to the target soundtrack.

[0033] In an embodiment of the present disclosure, a target video is displayed through a playback interface, a target object is a person's hand in the target video, and correspondingly, a target action is for the hand to touch a certain position on the playback interface, and optionally, when the target action touches a different position, the corresponding target soundtrack is different.

[0034] In an embodiment of the present disclosure, the target soundtrack corresponds to a target position on the playback interface, and optionally, when the target action is to touch a certain position on the playback interface, the audio processing method corresponding to the target action is determined to play the soundtrack corresponding to the position. In one embodiment of the present disclosure, when it is determined that the soundtrack corresponding to the upper left corner of the playback interface is a piece of music played on a guitar, and the target action is to touch the upper left corner of the playback interface, the music played on the guitar is added to the music and played.

[0035] An embodiment of the present disclosure aligns the soundtrack with the playback interface, and a target subject in the video touches the playback interface to trigger the soundtrack playback to create music.

[0036] The embodiments of the present disclosure provide one possible implementation, in which the target soundtrack is used to carry the audio of the target musical material, and the target position on the playback interface is marked with an identifier of the target musical material.

[0037] In an embodiment of the present disclosure, as shown in FIG. 2, the playback interface displays at least one music material identifier 202 on the playback interface 201, where one music material identifier represents a musical instrument, such as a piano, a drum set, a harmonica, etc., and the corresponding music identifier is determined based on the target action of the target object in the target video. In an embodiment of the present disclosure, the music material identifier displayed on the playback interface may be preset, and the preset music material identifier may be a fixed identifier of one or more music materials, or may be an identifier of one or more music materials automatically set based on different music or different music types. For example, the user selects the music or music type they want to edit, and the terminal device receives the user's selection operation and then displays the corresponding identifier of at least one music material based on the selected music or music type. Optionally, the music to be played may be a song pre-selected by the user or a pre-set song provided by the APP, and if the target action corresponds to a piano identifier in the playback interface, the piano soundtrack of the music is added to the music and played. Optionally, the user selects music, and a music selection interface displaying the music to be selected is displayed based on the user's music selection operation, and the user can select at least one of the music to be selected. The music material identifier may include the identifier of the corresponding music material of the soundtrack included in the music selected by the user, and the music material identifier may further include the identifier of another music material. Optionally, the user can click the identifiers of multiple music materials simultaneously, and the corresponding soundtracks of the multiple music materials are output.

[0038] The embodiment of the present disclosure displays the identifiers of music materials at the target positions of the playback interface, allowing users to select corresponding soundtracks of different music materials to process music according to their music needs.

[0039] The embodiments of the present disclosure provide one possible implementation form, in which the audio processing method specifically processes and then plays the music according to a target sound effect corresponding to the target action, the target action specifically being an action matching a preset posture, and the preset posture corresponding to the target sound effect.

[0040] In an embodiment of the present disclosure, specific special effects can be added to music by identifying target actions in the user video.

[0041] For ease of explanation of the embodiments of the present disclosure, one specific embodiment will be taken as an example. The target object may be a user's hand, mouth, head, eyeball, etc., and special musical effects may be added to music based on the target object's target movement. Optionally, specific processing may be performed on the music corresponding to the target object's corresponding musical material identifier based on the target object's movement. For example, when a user's hand lies on the display area of ​​a piano identifier, the fingers may perform a stretching motion, and the volume, tone, etc. of the piano soundtrack may be adjusted based on the stretching motion. Alternatively, if the target object is the user's mouth, a vibrato effect may be added to music based on the opening and closing of the user's mouth. For example, when the user's mouth is open, a vibrato effect may be added to music. Of course, the manner of adding special musical effects is not limited to the embodiments of the present disclosure.

[0042] The embodiments of the present disclosure add special musical effects to music through target actions, making music creation more diverse.

[0043] The embodiments of the present disclosure provide a possible implementation, in which a music material selection is displayed in a playback interface, as shown in FIG. 3, and the method includes: a step S301 of displaying a list of identifiers of candidate music materials when a trigger operation for selecting the music materials by a user is received; The method further includes step S302 of receiving a user's music material selection operation according to the list of candidate music material identifiers, and determining an identifier of at least one music material according to the candidate music material identifiers corresponding to the music material selection operation.

[0044] In embodiments of the present disclosure, the identifiers of the musical material displayed in the playback interface may be selected by the user.

[0045] For ease of explanation of the embodiments of the present disclosure, a specific embodiment will be taken as an example, as shown in FIG. 4. When a terminal device displays a playback interface 401, the playback interface 401 displays a music material selection option 402. Upon receiving a user's trigger operation for the music material selection option, the terminal device displays a list of identifiers for each candidate music material. The terminal device receives the user's music material selection operation through the list of candidate music material identifiers and determines the identifier of at least one music material based on the identifier of the music material corresponding to the music material selection operation. Specifically, based on the user's selection operation, the identifiers of all music materials selected by the user can be used as the music materials displayed in the music generation interaction interface. This may be based on the music materials included in the music selected by the user, assuming that the music selected by the user does not include all of the music materials included in the music. If the music materials selected by the user do not include all of the music materials included in the music, the identifiers of the music materials selected by the user can be displayed in the playback interface, and music materials not selected by the user but included in the music can also be displayed in the playback interface. To facilitate the description of the embodiments of the present disclosure, a specific embodiment will be taken as an example. When a terminal device responds to a user's music generation trigger operation, it activates a video capture device, captures a user video through the video capture device, and displays the user video on a playback interface. As shown in Fig. 5, a user video 502 and at least one music material identifier 503 are displayed on the playback interface 501, a target object 504 in the user video is identified, and an interaction between the target object 504 and the music material identifier 503 is obtained. Optionally, the interaction between the target object and the music material identifier can be the target object staying on the display area of ​​the music material identifier or the target object's movement on the display area of ​​the music material identifier, and music is generated based on the soundtrack corresponding to the music material identifier related to the interaction.

[0046] In the embodiment of the present disclosure, a music material selection option is displayed on the playback interface, and a music material selection operation by the user is received, and the identifier of the music material that needs to be displayed on the playback interface is determined based on the selection operation, so that the user can select different music materials according to their preferences to generate music, which is more personalized.

[0047] The embodiment of the present disclosure provides a possible implementation, in which if the duration of the touch action reaches a preset threshold, the touch action is determined as a target action.

[0048] In the embodiment of the present disclosure, whether a touch action of a target object is a target action is determined based on the duration of the touch action of the target object.

[0049] For ease of explanation of the embodiments of the present disclosure, a specific embodiment is taken as an example. When the target object is the user's hands in the user video, the time the user's hands remain stationary in the display area of ​​the music material identifier is used to determine whether to output the soundtrack corresponding to the music material identifier. Specifically, when the time the user's hands remain stationary in the display area of ​​the music material identifier is longer than 0.3 seconds, music is generated based on the soundtrack of the music material identifier. Of course, the time threshold can be set by those skilled in the art according to actual circumstances, and the present disclosure is not limited thereto.

[0050] The embodiment of the present disclosure provides a possible implementation, in which the step of generating music based on the soundtrack of the identifier of the corresponding target music material of the interactive operation includes: If there are at least two target sound effects corresponding to the target action at the same timing, the method includes a step of matching the beats of the at least two target sound effects and simultaneously reproducing the at least two matched target sound effects.

[0051] In the embodiment of the present disclosure, when there may be multiple objects of interactive operation of the target object at the same time, and there are multiple identifiers of musical materials that interact with the target object, the beats of the soundtracks corresponding to the identifiers of the multiple musical materials are synchronized, and the synchronized soundtracks are synthesized into music.

[0052] For ease of explanation of the embodiments of the present disclosure, a specific embodiment is taken as an example. A terminal device captures a user video using a video capture device, and then displays the user video on a playback interface. In this case, the target object in the user video is the user's hands, and the user's hands may simultaneously interact with two music material identifiers. In this case, the soundtracks corresponding to the identifiers of the two music materials are synchronized, and the two soundtracks after synchronization are combined into music. Specifically, when the user's hands interact with the piano and drum set identifiers simultaneously, the beats of the piano and drum set soundtracks are synchronized based on the start time of the user's interaction with the two identifiers. Specifically, when the user first interacts with the piano identifier, the corresponding piano soundtrack begins to be played. When the corresponding piano soundtrack is being played, if the user's hands also interact with the drum set identifier, the corresponding drum set soundtrack will be played from the next beat after the user's hands interact with the drum set in the piano soundtrack, thereby completing the synchronization of the beats of the two soundtracks and combining the synchronized two soundtracks into music.

[0053] In the embodiment of the present disclosure, when there are multiple musical material identifiers corresponding to the interactive operation at the same time, the beats of the music corresponding to the multiple musical material identifiers are matched, and the soundtrack after the beats are matched is synthesized into music, so that the user can interact with multiple musical material identifiers at the same time, and more diverse music is formed.

[0054] The embodiment of the present disclosure provides a music playing device based on user interaction, and as shown in FIG. 6, the music playing device based on user interaction 60 The eyes It may include a target video acquisition module 601, a target action identification module 602, and a music playback module 603. 、 The target video acquisition module 601 is used to acquire the target video by a video capture device; The target action identification module 602 is used to determine a corresponding audio processing manner of the target action when it is identified that the target object in the target video exhibits the target action; The music playing module 603 is used to play music according to the audio processing method.

[0055] Optionally, when the music playing module 603 plays music according to the audio processing method, the audio processing method specifically includes adding a target soundtrack corresponding to the target action to the music and playing it.

[0056] Optionally, if the target action identification module 602 identifies that the target object in the target video exhibits a target action, when determining a corresponding audio processing manner of the target action, the target action is specifically a touch action of the target object on a target position on the playback interface of the target video; The target position corresponds to the target soundtrack.

[0057] Optionally, the target soundtrack is used to carry the audio of the target musical material; The target position on the playback interface is marked with an identifier of the target musical material.

[0058] Optionally, when the music playing module 603 plays music according to the audio processing method, the audio processing method specifically processes the music according to the target sound effect corresponding to the target action before playing it.

[0059] Optionally, if the target action identification module 602 identifies that the target object in the target video exhibits a target action, when determining a corresponding audio processing manner for the target action, the target action is specifically a motion matching a preset posture; The preset pose corresponds to the target sound effect.

[0060] Optionally, the target action identification module 602 further comprises: upon receiving a user trigger action for selecting music material, displaying a list of identifiers for each candidate music material; The music material selection operation is received by the user through the list of candidate music material identifiers, and the target music material identifier is determined based on the candidate music material identifiers corresponding to the music material selection operation.

[0061] Alternatively, if the duration of the touching action reaches a preset threshold, the touching action is determined as a target action.

[0062] Optionally, the music playing module 603, when playing the music after processing it according to the corresponding target sound effect of the target action, If there are at least two target sound effects corresponding to the target action at the same timing, the timing is matched to the beats of the at least two target sound effects, and the matched beats of the at least two target sound effects are played back simultaneously.

[0063] The user interaction-based music playback device of the embodiment of the present disclosure can implement the user interaction-based music playback method shown in the above embodiment of the present disclosure, and the implementation principles are similar, so detailed descriptions are omitted here.

[0064] In the embodiment of the present disclosure, a playback interface is displayed based on the user's music generation trigger operation, a music material identifier is displayed on the playback interface, the user's interactive operation on the displayed music material identifier is obtained based on the playback interface, and music is generated based on the soundtrack of the target music material identifier corresponding to the interactive operation, so that the user can freely create personalized music based on their own operation, which does not restrict music creation, but also increases the interest of music creation and improves the user experience.

[0065] Referring now to FIG. 7, a structural diagram of an electronic device 700 suitable for implementing an embodiment of the present disclosure is shown. Examples of terminal devices in the embodiment of the present disclosure include, but are not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. The electronic device shown in FIG. 7 is merely an example and does not limit the functionality and scope of use of the embodiment of the present disclosure.

[0066] The electronic device includes a memory and a processor, which may be referred to as a processing unit 701 below, and the memory may include at least one of a read-only memory (ROM) 702, a random access memory (RAM) 703, and a storage device 708, specifically as follows:

[0067] 7, electronic device 700 may include a processing unit (e.g., a central processor, a graphics processor, etc.) 701, which can perform various appropriate operations and processes based on programs stored in read-only memory (ROM) 702 or programs uploaded from storage device 708 to random access memory (RAM) 703. RAM 703 further stores various programs and data necessary for the operation of electronic device 700. Processing unit 701, ROM 702, and RAM 703 are connected to each other via bus 704. Input / output (I / O) interface 705 is also connected to bus 704.

[0068] Typically, devices such as input devices 706, including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 707, including, for example, a liquid crystal display (LCD), speakers, oscillators, etc.; storage devices 708, including, for example, a magnetic tape, hard disk, etc.; and communication devices 709 may be connected to the I / O interface 705. The communication devices 709 enable the electronic device 700 to communicate wirelessly or via wires with other devices to exchange data. While FIG. 7 illustrates the electronic device 700 with a variety of devices, it should be understood that it is not required to implement or include all of the devices shown. More or fewer devices may alternatively be implemented or included.

[0069] In particular, according to embodiments of the present disclosure, the processes described with reference to the flowcharts may be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product, which includes a computer program embodied on a non-transitory computer-readable medium, the computer program including program code for performing the methods shown in the flowcharts. In such embodiments, the computer program may be downloaded and installed from a network via the communication device 709, or may be installed from the storage device 708, or may be installed from the ROM 702. When the computer program is executed by the processing device 701, it performs the functions defined in the methods of the embodiments of the present disclosure.

[0070] As described, the computer-readable medium of the present disclosure may be a computer-readable signal medium, a computer-readable storage medium, or any combination of the above. Computer-readable storage media include, but are not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in connection with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, and carrying computer-readable program code. Such propagated data signals may take various forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which is capable of transmitting, propagating, or transmitting a program used by or in connection with an instruction execution system, apparatus, or device. Program code contained in a computer-readable medium may be transmitted over any suitable medium, including, but not limited to, wire, cable, RF (radio frequency), etc., or any suitable combination of the above.

[0071] In some embodiments, clients and servers may communicate using any now known or later developed network protocol, such as HTTP (HyperText Transfer Protocol), and may be connected to one another via any form or medium of digital data communication (e.g., a communications network). Examples of communications networks include local area networks ("LANs"), wide area networks ("WANs"), the Internet (e.g., international networks), and end-to-end networks (e.g., ad hoc end-to-end networks), and any now known or later developed networks.

[0072] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated in the electronic device.

[0073] The computer-readable medium is loaded with one or more programs, and when the one or more programs are executed by the electronic device, the electronic device displays a playback interface displaying the identifier of at least one type of music material in response to a user's music generation trigger operation, obtains the user's interactive operation on the displayed music material identifier based on the playback interface, and generates music based on the soundtrack of the identifier of the target music material corresponding to the interactive operation.

[0074] Computer program code for carrying out the operations of the present disclosure can be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, and the like, and further including conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may run entirely on the user computer, partially on the user computer, as a separate software package, partially on the user computer and partially on a remote computer, or entirely on a remote computer or server. When referring to a remote computer, the remote computer may be connected to the user computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider).

[0075] The flowcharts and block diagrams in the drawings illustrate possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each box in a flowchart or block diagram may represent a module, program segment, or portion of code, which includes one or more executable instructions for implementing the specified logical function. It should be noted that in some alternative implementations, the functions marked in the boxes may be executed in an order different from the order marked in the drawings. For example, two boxes shown in succession may actually be executed substantially in parallel, but may also be executed in the reverse order, depending on the functionality involved. It should be noted that each box in the block diagrams and / or flowcharts, and combinations of boxes in the block diagrams and / or flowcharts, may be implemented in a dedicated system based on hardware that performs the specified functions or operations, or in a combination of dedicated hardware and computer instructions.

[0076] The relevant modules or units described in the embodiments of the present disclosure may be implemented in the form of software or hardware, and the names of the modules or units may in some cases not be intended to limit the units themselves.

[0077] The functions described herein may be implemented at least in part by one or more hardware logic components, including, but not limited to, exemplary types of hardware logic components that may be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), etc.

[0078] In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain or store a program used by or in connection with an instruction execution system, device, or apparatus. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the above. More specific examples of machine-readable storage media include one or more wire-based electrical connections, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0079] According to one or more embodiments of the present disclosure, a possible implementation is provided, providing a music playing method based on user interaction, the method comprising: collecting a target video with a video capture device; determining a corresponding audio processing manner of the target action when identifying that the target object exhibits the target action in the target video; and playing music according to the audio processing scheme.

[0080] Furthermore, the audio processing method specifically includes adding a target soundtrack corresponding to the target action to the music and playing it.

[0081] Furthermore, the target action is specifically a touch action of the target object on the playback interface of the target video at a target position on the playback interface; The target position corresponds to the target soundtrack.

[0082] Furthermore, the target soundtrack is used to carry the audio of the target music material; The target position on the playback interface is marked with an identifier of the target musical material.

[0083] Furthermore, the audio processing method specifically involves processing the music according to the target sound effect corresponding to the target action and then playing it back.

[0084] Furthermore, the target motion is specifically a motion that matches a preset posture, The preset pose corresponds to the target sound effect.

[0085] Furthermore, the method comprises: displaying a list of identifiers of each candidate musical material when a user trigger for musical material selection is received; The method further includes receiving a music material selection operation by a user according to the list of identifiers of each candidate music material, and determining an identifier of the target music material based on the identifiers of the candidate music materials corresponding to the music material selection operation.

[0086] Furthermore, if the duration of the touching motion reaches a preset threshold, the touching motion is determined as a target motion.

[0087] Furthermore, the step of processing and then playing the music in accordance with the target sound effect corresponding to the target action further comprises: If there are at least two target sound effects corresponding to the target action at the same timing, the method includes a step of matching the beats of the at least two target sound effects and simultaneously reproducing the at least two matched target sound effects.

[0088] According to one or more embodiments of the present disclosure, a possible implementation is provided, a target video acquisition module for acquiring the target video by a video capture device; a target action identification module for determining a corresponding audio processing manner of the target action when identifying that the target object exhibits a target action in the target video; and a music playing module for playing music according to the audio processing scheme.

[0089] Optionally, when the music playing module 603 plays music according to the audio processing method, the audio processing method specifically includes adding a target soundtrack corresponding to the target action to the music and playing it.

[0090] Optionally, if the target action identification module 602 identifies that the target object in the target video exhibits a target action, when determining a corresponding audio processing manner of the target action, the target action is specifically a touch action of the target object on a target position on the playback interface of the target video; The target position corresponds to the target soundtrack.

[0091] Optionally, the target soundtrack is used to carry the audio of the target musical material; The target position on the playback interface is marked with an identifier of the target musical material.

[0092] Optionally, when the music playing module 603 plays music according to the audio processing method, the audio processing method specifically processes the music according to the target sound effect corresponding to the target action before playing it.

[0093] Optionally, if the target action identification module 602 identifies that the target object in the target video exhibits a target action, when determining a corresponding audio processing manner for the target action, the target action is specifically a motion matching a preset posture; The preset pose corresponds to the target sound effect.

[0094] Optionally, the target action identification module 602 further comprises: upon receiving a user trigger action for selecting music material, displaying a list of identifiers for each candidate music material; The music material selection operation is received by the user through the list of candidate music material identifiers, and the target music material identifier is determined based on the candidate music material identifiers corresponding to the music material selection operation.

[0095] Alternatively, if the duration of the touching action reaches a preset threshold, the touching action is determined as a target action.

[0096] Optionally, the music playing module 603, when playing the music after processing it according to the corresponding target sound effect of the target action, If there are at least two target sound effects corresponding to the target action at the same timing, the timing is matched to the beats of the at least two target sound effects, and the matched beats of the at least two target sound effects are played back simultaneously.

[0097] According to one or more embodiments of the present disclosure, one or more processors; Memory and and one or more application programs stored in a memory and configured to be executed by one or more processors and configured to perform the user interaction based music playback method.

[0098] According to one or more embodiments of the present disclosure, a storage medium is provided, wherein the storage medium stores at least one instruction, at least one segment of a program, a code set, or an instruction set, and the at least one instruction, at least one segment of a program, a code set, or an instruction set is uploaded by a processor to perform the music playback method based on user interaction.

[0099] The above description merely describes preferred embodiments of the present disclosure and the technical principles used. As can be understood by those skilled in the art, the scope of the present disclosure is not limited to a technical solution consisting of a specific combination of the above technical features, but should also include other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the concept of the above disclosure. For example, technical solutions formed by replacing the above features with technical features having similar functions disclosed in the present disclosure (but not limited to these) can be included.

[0100] Also, although operations are described in a particular order, this should not be understood as requiring that these operations be performed in the particular order shown, or sequentially. In some cases, multitasking and parallel processing may be advantageous. Similarly, although the above discussion includes several specific implementation details, these should not be construed as limiting the scope of the present disclosure. Some features that are described in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment may also be implemented in multiple embodiments separately or in any suitable subcombination.

[0101] Although the present subject matter has been described in language specific to structural features and / or logical operations of a method, it will be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or operations described above. Rather, the specific features and operations described above are merely example forms of implementing the claims.

Claims

1. A music playback method based on user interaction, comprising: collecting a target video with a video capture device; determining a corresponding audio processing manner of the target action when identifying that the target object exhibits the target action in the target video; and playing music according to the audio processing scheme; the audio processing method includes a step of processing the music according to the target sound effect corresponding to the target action and then playing it back; the target motion includes a motion matching a preset posture, the preset posture corresponds to the target sound effect; the step of processing and then playing the music in accordance with the target sound effect corresponding to the target action, The music playback method based on user interaction includes a step of, if there are at least two target sound effects corresponding to the target action at the same timing, matching the beats of the at least two target sound effects and simultaneously playing the matched beats of the at least two target sound effects.

2. The method of claim 1 , wherein the audio processing method includes a step of adding a corresponding target soundtrack of the target action to the music and playing it.

3. the target action includes a touch action of the target object on a playback interface of the target video to a target position on the playback interface; The method of claim 2 , wherein the target location corresponds to the target soundtrack.

4. The target soundtrack is used to carry the audio of the target music material; 4. The method of claim 3, wherein the target position on the playback interface is marked with an identifier of the target musical material.

5. displaying a list of identifiers of each candidate musical material when a user trigger for musical material selection is received; 5. The method of claim 4, further comprising the step of: receiving a music material selection operation by a user with the list of identifiers of each candidate music material; and determining an identifier of the target music material based on the identifiers of the corresponding candidate music materials of the music material selection operation.

6. The method of claim 3 , further comprising determining the touch action as a target action if the duration of the touch action reaches a preset threshold.

7. A user interaction-based music playback device, comprising: a target video acquisition module for acquiring the target video by a video capture device; a target action identification module for determining a corresponding audio processing manner of the target action when identifying that the target object exhibits a target action in the target video; a music playback module for playing music according to the audio processing method; the audio processing method includes a step of processing the music according to the target sound effect corresponding to the target action and then playing it back; the target motion includes a motion matching a preset posture, the preset posture corresponds to the target sound effect; the step of processing and then playing the music in accordance with the target sound effect corresponding to the target action, A music playback device based on user interaction, comprising a step of, if there are at least two target sound effects corresponding to the target action at the same timing, matching the beats of the at least two target sound effects and simultaneously playing the matched beats of the at least two target sound effects.

8. A device, one or more processors; Memory and one or more application programs stored in the memory and configured to be executed by the one or more processors, and configured to perform the user interaction based music playback method of any one of claims 1 to 6.

9. A storage medium having stored thereon at least one instruction, at least one segment of a program, a code set or an instruction set, said at least one instruction, said at least one segment of a program, said code set or instruction set being uploaded and executed by a processor to realize the music playback method based on user interaction according to any one of claims 1 to 6.

Images