Method and system for playing a game based on a multimedia file

Abstract

The embodiment of the present application provides a game method based on a multimedia file, which comprises the following steps: playing the multimedia file; separating the multimedia file to obtain a target audio signal; obtaining a plurality of rhythm points in the target audio signal; respectively configuring a corresponding interactive component for each rhythm point; monitoring the playing progress of the multimedia file; and when the playing progress reaches a time node corresponding to one of the rhythm points, displaying a target interactive component on a playing interface of the multimedia file; wherein the target interactive component is the interactive component corresponding to the rhythm point. In the embodiment of the present application, the corresponding interactive component can be popped up at the playing moment of the audio content at the audio rhythm point, so as to be interactively operated by the user. That is, the computer device effectively increases the interactive mode of the audio game in the process of playing the multimedia file, and improves the experience of the user when watching the multimedia file.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a game method, system, computer device, and computer-readable storage medium based on multimedia files. Background Technology

[0002] Users are increasingly interacting with computer devices such as tablets and mobile phones, for example, by watching movies or playing games. With the popularization of concepts like augmented reality and virtual reality, people's demands for the entertainment capabilities of computer devices are rising. However, current computer devices offer limited output formats and few interactive methods when playing movies, stand-up comedy, and other video content, resulting in a poor user experience and clearly failing to meet current entertainment needs. Summary of the Invention

[0003] The purpose of this application is to provide a game method, system, computer device, and computer-readable storage medium based on multimedia files to solve the following problems: In the prior art, when a computer device plays multimedia files, the output format is limited, the interactive methods are few, and the user experience is poor.

[0004] One aspect of this application provides a game method based on a multimedia file, the method comprising: playing a multimedia file; separating the multimedia file to obtain a target audio signal; acquiring multiple rhythm points in the target audio signal; configuring a corresponding interactive component for each rhythm point; monitoring the playback progress of the multiple media files; and when the playback progress reaches a time node corresponding to one of the rhythm points, displaying a target interactive component on the playback interface of the multimedia file; wherein the target interactive component is an interactive component corresponding to this rhythm point.

[0005] Optionally, separating the multimedia file to obtain the target audio signal includes: extracting the initial audio signal of the multimedia file; determining the music audio signal and non-music audio signal in the initial audio signal; and separating the initial audio signal to obtain the target audio signal corresponding to the music audio signal.

[0006] Optionally, acquiring multiple rhythm points in the target audio signal includes: detecting the target audio signal to acquire multiple musical drumbeats from the target audio signal based on the detection result; and determining each audio drumbeat as a corresponding rhythm point.

[0007] Optionally, acquiring multiple rhythm points in the target audio signal includes: acquiring the lyrics text of the target audio file; and predicting the vocalization point of each lyric in the lyrics text, so as to determine the vocalization point of each lyric as a corresponding rhythm point.

[0008] Optionally, obtaining multiple rhythm points in the target audio signal includes: identifying the music style of the target audio file; and finding the multiple rhythm points from the target audio signal using preset rules adapted to the music style.

[0009] Optionally, a corresponding interactive component is configured for each rhythm point, including: determining the interaction type of the interactive component of the i-th rhythm point based on the time interval between the i-th rhythm point and a preset number of neighboring rhythm points; where i is a positive integer ≤ N, and N is the number of the plurality of rhythm points.

[0010] Optionally, a corresponding interactive component is configured for each rhythm point, including: determining the spatial position of the interactive component of the i-th rhythm point according to the time node of the i-th rhythm point and the interaction type of the corresponding interactive component; where i is a positive integer ≤ N, and N is the number of the multiple rhythm points.

[0011] Optionally, the multimedia file is a video file; displaying the target interactive component on the playback interface of the multimedia file includes: popping the target interactive component into a target area, wherein the target area is the background area of ​​the video frame of the video file.

[0012] One aspect of this application provides a game system based on multimedia files, the game system comprising: a playback module for playing multimedia files; a separation module for separating the multimedia files to obtain a target audio signal; an acquisition module for acquiring multiple rhythm points in the target audio signal; a configuration module for configuring a corresponding interactive component for each rhythm point; a monitoring module for monitoring the playback progress of the multiple media files; and an interaction module for displaying a target interactive component on the playback interface of the multimedia files when the playback progress reaches a time node corresponding to one of the rhythm points; wherein the target interactive component is an interactive component corresponding to this rhythm point.

[0013] One aspect of this application provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the game method based on multimedia files described above.

[0014] One aspect of this application provides a computer-readable storage medium, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the game method based on multimedia files described above.

[0015] The game method, system, device, and computer-readable storage medium based on multimedia files provided in this application enable computer devices to pop up corresponding interactive components at the playback points of the audio content during the playback of multimedia files, allowing users to interact with them. In other words, the computer device effectively increases the interactivity of audio games and enhances the user experience when watching multimedia files. Attached Figure Description

[0016] Figure 1 This diagram schematically illustrates an application environment for a game method based on multimedia files according to an embodiment of this application.

[0017] Figure 2 A flowchart illustrating a multimedia file-based game method according to Embodiment 1 of this application is shown schematically.

[0018] Figure 3 for Figure 2 Flowchart of the sub-steps in step S202;

[0019] Figure 4 for Figure 2 Flowchart of the sub-steps in step S204;

[0020] Figure 5 for Figure 2 Flowchart of another sub-step in step S204;

[0021] Figure 6 for Figure 2 Flowchart of another sub-step in step S204;

[0022] Figure 7 This illustration schematically shows another flowchart of a multimedia file-based game method according to Embodiment 1 of this application;

[0023] Figure 8 This illustration schematically shows another flowchart of a multimedia file-based game method according to Embodiment 1 of this application;

[0024] Figure 9 This illustration schematically shows another flowchart of a multimedia file-based game method according to Embodiment 1 of this application;

[0025] Figure 10 A block diagram of a multimedia file-based game system according to Embodiment 2 of this application is schematically shown; and

[0026] Figure 11 The illustration shows a schematic diagram of the hardware architecture of a computer device suitable for implementing a game method based on multimedia files, according to Embodiment 3 of this application. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0028] It should be noted that the descriptions involving "first," "second," etc., in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0029] In the description of this application, it should be understood that the numerical labels before the steps do not indicate the order of the steps, but are only used to facilitate the description of this application and to distinguish each step, and therefore should not be construed as a limitation of this application.

[0030] Computer device 2 can be configured to provide multimedia playback services. The multimedia playback services can be configured to receive content via various transmission technologies and play the received content. The content may include content streams (e.g., video streams, audio streams), content files (e.g., video files, audio files), and / or other data.

[0031] Computer device 2 can be configured to access the content and services of provider network 4. Computer device 2 can be a mobile device such as a smartphone, tablet, or augmented reality / virtual reality device. Computer device 2 can be associated with one or more users. A single user can use computer device 2 to access provider network 4. Computer device 2 can travel to various locations and use different networks to access provider network 4.

[0032] Computer device 2 may include application 2A and other applications. Application 2A outputs content (e.g., displays, renders, presents) to the user. Application 2A may be a multimedia player (such as a bilibili client), a browser client, etc. As an example, Figure 1 The video shows the screen when the application 2A plays a cartoon video.

[0033] like Figure 1As shown, in audio game mode, computer device 2 can perform the following operations: generate different interactive components (e.g., Z1) in real time based on the video file, and use the video frame of the video file as the game background of the interactive component (e.g., Z1), so that the user can experience audio games while watching the video frame of the video file, thereby achieving a multi-faceted experience of visual, auditory, and human-computer interaction, and increasing user interest. It should be understood that... Figure 1 The video scene shown is exemplary and is not intended to limit the scope of protection of this application.

[0034] Example 1

[0035] Figure 2 A flowchart illustrating a game method based on multimedia files according to Embodiment 1 of this application is shown schematically. The following exemplary description uses computer device 2 as the execution subject. Figure 2 As shown, the game method based on multimedia files may include steps S200 to S210, wherein:

[0036] Step S200: Play the multimedia file.

[0037] The multimedia files can include various types of audio and video content, such as music, crosstalk, or other types of content. For music, it can include instrumental music without lyrics or music with lyrics.

[0038] The multimedia file may be a local file on computer device 2 or an online file obtained through the Internet.

[0039] Step S202: Separate the multimedia file to obtain the target audio signal.

[0040] To improve the accuracy of detecting rhythm points in the multimedia file, the target audio signal is an audio signal without noise interference or with noise interference controlled within a preset range, or an audio signal after removing noise.

[0041] Which signals in the multimedia file can be considered noise signals is determined based on the source of the rhythm points. Different types of content have different user preferences for rhythm, and therefore the sources of the rhythm points may also differ. In an exemplary embodiment, the source of the rhythm points can be determined based on the multimedia file's category, description, comment keywords, etc. For example, the rhythm of music-related audio-visual content often comes from musical beats, so its rhythm points can be the drumbeats or the vocalizations of lyrics. The rhythm of crosstalk-related audio-visual content often comes from the pronunciation rhythm of the crosstalk performers, so its rhythm points can be the vocalizations of each word or the vocalizations with emphasis.

[0042] As an example, the multimedia file is music-related audio and video content. Computer device 2 can find rhythm points from the music audio signal in the multimedia file. That is, computer device 2 can use the music audio signal as the target audio signal and the non-music audio signal as the noise signal. In subsequent operations, interactive components that interact with the user are generated based on these rhythm points, allowing the user to interact in sync with the audio beat. Figure 3 As shown, step S202 may include steps S300 to S304, wherein: step S300, extracting the initial audio signal of the multimedia file; step S302, determining the music audio signal and non-music audio signal in the initial audio signal; and step S304, separating the initial audio signal to obtain the target audio signal corresponding to the music audio signal. In some embodiments, the computer device 2 can separate the music audio signal and non-music audio signal in the initial audio signal using a trained deep neural network model, etc. Of course, the computer device 2 can also implement the separation operation in step S402 in other ways, such as time-domain or frequency-domain analysis.

[0043] Step S204: Obtain multiple rhythm points in the target audio signal.

[0044] The multiple rhythm points can be musical drumbeats, the vocalization points of lyrics, or the vocalization points of each word in a crosstalk performance.

[0045] (1) The multiple rhythm points mentioned are musical drumbeats.

[0046] To effectively and quickly detect the multiple rhythm points, such as Figure 4 As shown, step S204 may include steps S400 to S402, wherein: step S400, detecting the target audio signal to obtain multiple musical drum beats from the target audio signal according to the detection result; step S402, determining each audio drum beat as a corresponding rhythm point.

[0047] The musical drumbeats can be strong drumbeats and / or weak drumbeats, and can be audio segments or audio points whose energy change amplitude exceeds a first threshold, whose pitch change amplitude exceeds a second threshold, or whose timbre change amplitude exceeds a third threshold.

[0048] Computer device 2 can perform the recognition operation in step S400 using a trained deep neural network model, time-domain analysis, or frequency-domain analysis. The deep neural network model identifies the multiple musical drumbeats by analyzing the audio features in the target audio signal. The time-domain analysis finds the multiple musical drumbeats by calculating the cumulative energy of the waveform file. The frequency-domain analysis finds the multiple musical drumbeats by calculating the changes in spectral energy.

[0049] (2) The multiple rhythm points mentioned are the vocal points of the lyrics.

[0050] To effectively and quickly detect the multiple rhythm points, such as Figure 5 As shown, step S204 may include steps S500 to S502, wherein: step S500, obtaining the lyrics text of the target audio file; step S502, predicting the vocalization point of each lyric in the lyrics text, so as to determine the vocalization point of each lyric as the corresponding rhythm point.

[0051] The point at which each lyric is uttered corresponds to the moment when each word or stressed word in the lyric begins to be pronounced.

[0052] Computer device 2 can predict the probability of each word being stressed, long, etc., when sung based on lyrics. Taking the lyrics "the crescent moon" as an example, if the probability of "bright" being stressed when sung is predicted to be 99%, then the sound produced when "bright" is sung is identified as a vocal point, and the time point of this vocal point can be the moment when "bright" begins to be sung.

[0053] Computer device 2 can predict the vocalization point of each lyric using various models such as trained deep neural network models and long short-term memory networks.

[0054] (3) The multiple rhythm points mentioned are the vocal points of each word in the process of performing a crosstalk program.

[0055] As mentioned earlier, if the multimedia file is audio-visual content in the form of crosstalk, the user will focus more on the rhythm of the speech.

[0056] Therefore, when the multimedia file is audio-visual content of the crosstalk genre, the target audio signal is a human voice signal. That is, the computer device 2 can use the pronunciation point of each word as a rhythm point, or the pronunciation point with emphasis as a rhythm point.

[0057] The above only provides a few methods for obtaining rhythm points in different situations, and is not intended to limit the scope of protection of this application.

[0058] To effectively find rhythmic points that match the intended mood, when the multimedia file is music-related audio-visual content, computer device 2 can search for rhythmic points based on the music style. For example... Figure 6 As shown, step S204 may include steps S600 to S602, wherein: step S600 identifies the music style of the target audio file; step S602 identifies the multiple rhythm points from the target audio signal using preset rules adapted to the music style. It should be noted that the preset rules corresponding to different music styles can be pre-defined or obtained through training on a massive dataset of audio files of the same style.

[0059] Step S206: Configure the corresponding interactive components for each rhythm point.

[0060] The interactive components can be various types of touch-based interactive components, such as tap-type interactive components, long-press interactive components, and swipe-type interactive components. Each type of interactive component can include components of various shapes, colors, etc.

[0061] Each rhythm point corresponds to at least one interactive component. The type and spatial location of the interactive component corresponding to each node can be determined in the following way:

[0062] In an exemplary embodiment, such as Figure 7 As shown, step S206 may include step S700: determining the interaction type of the interactive component of the i-th rhythm point based on the time interval between the i-th rhythm point and a preset number of neighboring rhythm points; where i is a positive integer ≤ N, and N is the number of the plurality of rhythm points. As an example, for rhythm points with sparse rhythms, a touch-type interactive component can be configured; for rhythm points with dense rhythms and parts where the singer has sustained notes, a long-press interactive component can be used, and then different forms of strategy combinations based on music style, etc., can be used to achieve diversity and fun in the interaction.

[0063] In an exemplary embodiment, such as Figure 8 As shown, step S206 may include step S800: determining the spatial position of the interactive component at the i-th rhythm point based on the time node of the i-th rhythm point and the interaction type of the corresponding interactive component; where i is a positive integer ≤ N, and N is the number of the plurality of rhythm points. As an example, the computer device 2 can generate the spatial positions of each interactive component according to different preset curve fitting distributions. The spatial position may be the relative position of the corresponding interactive component in the playback interface. The spatial positions of all interactive components can be stored in an audio game map, which may include various relevant information such as the relative position of each interactive component in the playback interface and the longest continuous display time. It should be understood that the preset curve fitting distribution is obtained by fitting according to the time interval of the rhythm points, the type of the rhythm points, and / or a set rule.

[0064] Of course, the spatial position of the interactive component at the i-th rhythm point can also be determined based on the playback screen.

[0065] Step S208: Monitor the playback progress of the multiple media files.

[0066] The playback progress refers to the current playback time point corresponding to the currently playing multimedia content (video frame). The playback progress can refer to the current playback time point as a relative position point or a percentage of progress within the total playback time interval.

[0067] Step S210: When the playback progress reaches a time node corresponding to one of the rhythm points, a target interactive component is displayed on the playback interface of the multimedia file; wherein, the target interactive component is the interactive component corresponding to this rhythm point.

[0068] The target interactive component can generate corresponding responses based on the user's touch operation in the touch area, thereby realizing human-computer interaction.

[0069] Given that the spatial location of the target interactive component has been determined, the computer device 2 can perform the following operations: First, display the target interactive component on the playback interface of the multimedia file according to the determined spatial location; second, monitor the user's touch operation on the target interactive component; third, generate a corresponding response according to the interaction logic of the target interactive component. The corresponding response may include: dynamic changes or disappearance of the target interactive component on the playback interface.

[0070] like Figure 9 As shown, when the spatial location of the target interactive component is not determined, computer device 2 can execute step S900: pop the target interactive component into a target area, where the target area is the background area of ​​the video frame of the video file. For example, if the multimedia file is a video file, and the pop-up time of the target interactive component is 52 minutes and 18 seconds, with a maximum duration of 0.5 seconds, this means that the target interactive component may be displayed on the video frame between 52 minutes and 18 seconds and 52 minutes and 18.5 seconds. In this case, if the target interactive component covers the main image (such as a face, animal, etc.), it will affect the video viewing experience. Therefore, computer device 2 can detect all video frames or several first-row video frames between 52 minutes and 18 seconds and 52 minutes and 18.5 seconds in the video file, find the common background area of ​​these video frames as much as possible, and display the target interactive component in that background area. The advantage of this is that it ensures the user's visual viewing experience as much as possible while simultaneously allowing for audio interactive games, greatly improving the user experience.

[0071] The multimedia file-based game method provided in this embodiment enables a computer device to automatically detect various rhythm points in the multimedia file during playback, temporarily configure corresponding interactive components for each rhythm point, and pop up the corresponding interactive components on the playback interface when the multimedia file reaches the time node corresponding to each rhythm point for user interaction. In other words, even when the multimedia file itself does not provide interactivity, this embodiment can temporarily configure different interactive components based on the multimedia content. This allows for the output of visual and auditory information while dynamically displaying temporarily shown interactive components at different time points. This enables real-time interaction with the user on the multimedia file playback interface through the pop-up interactive components, improving the user experience and broadening its applicability.

[0072] In this embodiment, the computer device 2 can generate interactive components in real time according to the current playback content, which effectively increases the interactive methods of audio games, thereby realizing visual, auditory and human-computer interaction and improving the user experience.

[0073] Example 2

[0074] Figure 10 The diagram illustrates a block diagram of a multimedia file-based game system according to Embodiment 2 of this application. This multimedia file-based game system can be divided into one or more program modules. One or more program modules are stored in a storage medium and executed by one or more processors to complete the embodiment of this application. The program module referred to in this embodiment is a series of computer program instruction segments capable of performing a specific function. The following description will specifically introduce the function of each program module in this embodiment.

[0075] like Figure 10 As shown, the multimedia file-based game system 1000 may include a playback module 1010, a separation module 1020, an acquisition module 1030, a configuration module 1040, a monitoring module 1050, and an interaction module 1060, wherein:

[0076] Playback module 1010 is used to play multimedia files;

[0077] Separation module 1020 is used to separate the multimedia file to obtain the target audio signal;

[0078] The acquisition module 1030 is used to acquire multiple rhythm points in the target audio signal;

[0079] Configuration module 1040 is used to configure corresponding interactive components for each rhythm point;

[0080] Monitoring module 1050 is used to monitor the playback progress of the multiple media files; and

[0081] The interactive module 1060 is used to display a target interactive component on the playback interface of the multimedia file when the playback progress reaches a time node corresponding to one of the rhythm points; wherein, the target interactive component is the interactive component corresponding to this rhythm point.

[0082] In an exemplary embodiment, the separation module 1020 is further configured to: extract the initial audio signal of the multimedia file; determine the music audio signal and non-music audio signal in the initial audio signal; and separate the initial audio signal to obtain the target audio signal corresponding to the music audio signal.

[0083] In an exemplary embodiment, the acquisition module 1030 is further configured to: detect the target audio signal to acquire multiple musical drumbeats from the target audio signal based on the detection result; and determine each audio drumbeat as a corresponding rhythm point.

[0084] In an exemplary embodiment, the acquisition module 1030 is further configured to: acquire the lyrics text of the target audio file; and predict the vocalization point of each lyric in the lyrics text, so as to determine the vocalization point of each lyric as a corresponding rhythm point.

[0085] In an exemplary embodiment, the acquisition module 1030 is further configured to: identify the music style of the target audio file; and find the plurality of rhythm points from the target audio signal by means of a preset rule adapted to the music style.

[0086] In an exemplary embodiment, the configuration module 1040 is further configured to: determine the interaction type of the interactive component of the i-th rhythm point based on the time interval between the i-th rhythm point and a preset number of neighboring rhythm points; wherein i is a positive integer ≤ N, and N is the number of the plurality of rhythm points.

[0087] In an exemplary embodiment, the configuration module 1040 is further configured to: determine the spatial position of the interactive component of the i-th rhythm point according to the time node of the i-th rhythm point and the interaction type of the corresponding interactive component; wherein i is a positive integer ≤ N, and N is the number of the plurality of rhythm points.

[0088] In an exemplary embodiment, the multimedia file is a video file; the interactive module 1060 is further configured to: pop the target interactive component into a target area, the target area being the background area of ​​the video frame of the video file.

[0089] Example 3

[0090] Figure 11This illustration schematically shows a hardware architecture diagram of a computer device 2 suitable for implementing a game method based on multimedia files according to Embodiment 3 of this application. In this embodiment, the computer device 2 is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions. For example, it can be a smartphone, tablet computer, laptop computer, desktop computer, rack server, blade server, tower server, or cabinet server (including independent servers or server clusters composed of multiple servers), etc. Figure 11 As shown, computer device 2 includes, but is not limited to: a memory 1110, a processor 1120, a network interface 1130, and a vibration element 1140 that can communicate with each other via a system bus. Wherein:

[0091] The memory 1110 includes at least one type of computer-readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 1110 may be an internal storage module of the computer device 2, such as the hard disk or memory of the computer device 2. In other embodiments, the memory 1110 may also be an external storage device of the computer device 2, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. Of course, the memory 1110 may include both the internal storage module and the external storage device of the computer device 2. In this embodiment, the memory 1110 is typically used to store the operating system and various application software installed on the computer device 2, such as program code for game methods based on multimedia files. In addition, the memory 1110 can also be used to temporarily store various types of data that have been output or will be output.

[0092] In some embodiments, processor 1120 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. Processor 1120 is typically used to control the overall operation of computer device 2, such as performing control and processing related to data interaction or communication with computer device 2. In this embodiment, processor 1120 is used to run program code stored in memory 1110 or process data.

[0093] Network interface 1130 may include a wireless network interface or a wired network interface, which is typically used to establish a communication link between computer device 2 and other computer devices. For example, network interface 1130 is used to connect computer device 2 to an external terminal via a network, establishing a data transmission channel and communication link between computer device 2 and the external terminal. The network may be an intranet, the Internet, Global System for Mobile Communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi, or other wireless or wired networks.

[0094] It should be pointed out that, Figure 11 Only computer devices with components 1110-1130 are shown; however, it should be understood that it is not required to implement all of the shown components, and more or fewer components may be implemented instead.

[0095] In this embodiment, the game method based on multimedia files stored in memory 1110 can also be divided into one or more program modules and executed by one or more processors (processor 1120 in this embodiment) to complete the embodiment of this application.

[0096] Example 4

[0097] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the multimedia file-based game method in the embodiments.

[0098] In this embodiment, the computer-readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the computer-readable storage medium can be an internal storage unit of a computer device, such as the hard disk or memory of the computer device. In other embodiments, the computer-readable storage medium can also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device. Of course, the computer-readable storage medium can also include both the internal storage unit and the external storage device of the computer device. In this embodiment, the computer-readable storage medium is typically used to store the operating system and various application software installed on the computer device, such as the program code of the game method based on multimedia files in this embodiment. In addition, the computer-readable storage medium can also be used to temporarily store various types of data that have been output or will be output.

[0099] Obviously, those skilled in the art should understand that the modules or steps of the embodiments of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the embodiments of this application are not limited to any particular combination of hardware and software.

[0100] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A game method based on multimedia files, characterized in that, The method includes: Play multimedia files; The multimedia file is separated to obtain the target audio signal; Obtain multiple rhythm points in the target audio signal; Configure corresponding interactive components for each rhythm point; Monitor the playback progress of the multimedia file; and When the playback progress reaches a time node corresponding to one of the rhythm points, a target interactive component is displayed on the playback interface of the multimedia file; wherein, the target interactive component is an interactive component corresponding to this rhythm point; the target interactive component is used to generate a corresponding response based on the user's touch operation in the touch area; Each rhythm point is configured with corresponding interactive components, including: The interaction type of the interactive component of the i-th rhythm point is determined based on the time interval between the i-th rhythm point and a preset number of neighboring rhythm points; where i is a positive integer ≤ N, and N is the number of the plurality of rhythm points.

2. The game method based on multimedia files according to claim 1, characterized in that, Separating the multimedia file to obtain the target audio signal includes: Extract the initial audio signal from the multimedia file; Determine the music audio signal and non-music audio signal in the initial audio signal; and The initial audio signal is separated to obtain the target audio signal corresponding to the music audio signal.

3. The game method based on multimedia files according to claim 2, characterized in that, Obtaining multiple rhythm points in the target audio signal includes: The target audio signal is detected to obtain multiple musical drum beats from the target audio signal based on the detection results; and Each audio drum beat is identified as a corresponding rhythm point.

4. The game method based on multimedia files according to claim 2, characterized in that, Obtaining multiple rhythm points in the target audio signal includes: Obtain the lyrics text of the target audio signal; and Predict the vocalization point of each lyric in the lyrics text, so as to determine the vocalization point of each lyric as the corresponding rhythm point.

5. The game method based on multimedia files according to claim 2, characterized in that, Obtaining multiple rhythm points in the target audio signal includes: Identify the musical style of the target audio signal; and The multiple rhythm points are identified from the target audio signal using preset rules adapted to the music style.

6. The game method based on multimedia files according to any one of claims 1 to 5, characterized in that, Configure corresponding interactive components for each rhythm point, including: Based on the time node of the i-th rhythm point and the interaction type of the corresponding interactive component, the spatial position of the interactive component of the i-th rhythm point is determined; where i is a positive integer ≤ N, and N is the number of the multiple rhythm points.

7. The game method based on multimedia files according to any one of claims 1 to 5, characterized in that, The multimedia file is a video file; Displaying target interactive components on the playback interface of the multimedia file, including: The target interactive component is popped up in the target area, which is the background area of ​​the video frame of the video file.

8. A game system based on multimedia files, characterized in that, include: The playback module is used to play multimedia files. A separation module is used to separate the multimedia file to obtain the target audio signal; The acquisition module is used to acquire multiple rhythm points in the target audio signal; The configuration module is used to configure the corresponding interactive components for each rhythm point; The monitoring module is used to monitor the playback progress of the multimedia file; and An interactive module is used to display a target interactive component on the playback interface of the multimedia file when the playback progress reaches a time node corresponding to a certain rhythm point; wherein, the target interactive component is an interactive component corresponding to this rhythm point; the target interactive component is used to generate a corresponding response based on the user's touch operation in the touch area; The configuration module is further configured to determine the interaction type of the interactive component of the i-th rhythm point based on the time interval between the i-th rhythm point and a preset number of neighboring rhythm points; where i is a positive integer ≤ N, and N is the number of the plurality of rhythm points.

9. A computer device, the computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the game method based on multimedia files as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that can be executed by at least one processor to cause the at least one processor to perform the steps of the game method based on a multimedia file as described in any one of claims 1 to 7.

11. A computer program product comprising computer instructions, characterized in that, When executed by a processor, the computer instructions implement the steps of the game method based on multimedia files as described in any one of claims 1 to 7.

Images