A multimedia data processing method, apparatus, device, and medium

Abstract

Embodiments of the present application provide a multimedia data processing method, device and equipment and medium, wherein the method can include: obtaining multimedia data to be played, the multimedia data including N video and audio material resources and configuration information of each video and audio material resource configured based on an animation protocol, the configuration information of a target material resource in the N video and audio material resources including interaction event information, N being an integer greater than 1; calling an animation playing engine based on the animation protocol to perform parsing and playing processing on the multimedia data; wherein, during the parsing and playing processing, if the target material resource is parsed, the target material resource is processed according to an interaction event corresponding to the monitored interaction event information. The embodiments of the present application can improve the flexibility of multimedia data creation.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, device and medium for processing multimedia data. Background Technology

[0002] With the rapid development of computer technology, multimedia data (such as video, audio, and animation) has become increasingly popular due to its richness and vividness. However, practice has shown that current multimedia data production methods involve simply piecing together existing material resources (such as images, text, and icons). For example, current video production involves combining image frames and text resources. This method of multimedia data production is relatively simple and fixed, resulting in low flexibility. Summary of the Invention

[0003] This application provides a method, apparatus, device, and medium for processing multimedia data, which can improve the flexibility of multimedia data creation.

[0004] On one hand, embodiments of this application provide a method for processing multimedia data, the method comprising: Obtain multimedia data to be played, the multimedia data including N audio-visual material resources and configuration information obtained by configuring each audio-visual material resource based on the animation protocol, the configuration information of the target audio-visual material resource among the N audio-visual material resources includes interactive event information, where N is an integer greater than 1; The animation playback engine based on the animation protocol is invoked to parse the multimedia data, and the various audio-visual materials in the multimedia data are played according to the parsing results; If the target audio-visual material resource is parsed, the target audio-visual material resource is played according to the parsing result, and the interaction event corresponding to the interaction event information is monitored. If the interaction event is detected, the next audio-visual material resource in the multimedia data is parsed.

[0005] On the other hand, embodiments of this application provide a multimedia data processing apparatus, the apparatus comprising: The acquisition unit is used to acquire multimedia data to be played. The multimedia data includes N audio-visual material resources and configuration information obtained by configuring each audio-visual material resource based on the animation protocol. The configuration information of the target material resource among the N audio-visual material resources includes interactive event information, where N is an integer greater than 1. The processing unit is used to call the animation playback engine based on the animation protocol to parse and play the multimedia data; During the parsing and playback process, if the target material resource is parsed, the target material resource is played according to the interaction event corresponding to the monitored interaction event information.

[0006] On the other hand, embodiments of this application provide a computer device, the computer device comprising: Storage device for storing computer applications; The processor executes computer applications stored in the storage device to implement the multimedia data processing method described above.

[0007] On the other hand, embodiments of this application provide a computer-readable storage medium storing a computer application program that, when executed, implements the multimedia data processing method described above.

[0008] On the other hand, embodiments of this application provide a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the aforementioned multimedia data processing method.

[0009] In this embodiment, an animation protocol can be used to configure the N audio-visual resources contained in the multimedia data, obtaining the configuration information for each audio-visual resource. This enables code-free creation of multimedia data, improving the efficiency of multimedia data creation. Furthermore, the configuration information for the target resource includes interactive event information, enriching the playback effects of the created multimedia data. Additionally, users can create multimedia data through the animation protocol, allowing each user to create multimedia data according to their own scenario needs, thus improving the flexibility of multimedia data creation. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This illustration shows a flowchart of playing an interactive animation provided by an exemplary embodiment of this application; Figure 2 A flowchart illustrating a multimedia data processing method provided in an exemplary embodiment of this application is shown. Figure 3a This illustration shows a schematic diagram of an interactive event including an erase event, provided by an exemplary embodiment of this application, in which multimedia data is played. Figure 3b This illustration shows a schematic diagram of an interactive event including an erase event, provided by an exemplary embodiment of this application, in which multimedia data is played. Figure 4 This illustration shows a schematic diagram of playing target material resources when the interactive event information is external data acquisition event information, according to an exemplary embodiment of this application. Figure 5 This illustration shows a schematic diagram of playing target material resources when the interactive event information is interactive processing event information, according to an exemplary embodiment of this application; Figure 6 A flowchart illustrating a multimedia data processing method provided in an exemplary embodiment of this application is shown. Figure 7 This illustration shows a schematic diagram of selecting audio-visual material resources by dragging a component, according to an exemplary embodiment of this application. Figure 8 This illustration shows an architectural diagram of an animation playback engine based on an animation protocol, provided in an exemplary embodiment of this application. Figure 9 This illustration shows a flowchart of a multimedia data parsing process using an animation playback engine, provided by an exemplary embodiment of this application. Figure 10a This illustration shows a flowchart of a multimedia data parsing process using an animation playback engine, provided by an exemplary embodiment of this application. Figure 10b This illustration shows a schematic diagram of an interface for playing multimedia data, provided by an exemplary embodiment of this application. Figure 11 This invention provides a schematic diagram of the structure of a multimedia data processing apparatus according to an exemplary embodiment of the present application. Figure 12 This illustration shows an architectural diagram of a computer device provided in an exemplary embodiment of this application. Detailed Implementation

[0012] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0013] This application relates to multimedia data, which may include, but is not limited to, animation, audio, images, text, video, etc. Animation can refer to a video stream containing multiple frames, producing a continuous animation effect when played. This continuous animation effect is based on the visual principle of the human eye, which states that once the human eye sees an object, that object remains visible for 0.34 seconds. Playing images frame by frame using this principle creates a smooth visual change, thus achieving a continuous animation effect. Animations can be made into interactive animations (or simply interactive animations). Specifically, adding interactive features to an animation creates an interactive animation. During the playback of an interactive animation, continuous playback is only possible when an interactive event is detected. Interactive events may include events generated when the interactive animation interacts with the user, or events generated when the interactive animation interacts with the background, etc. The interactive events involved in the embodiments of this application may include: events generated when elements in the playback interface that trigger the playback of the animation (such as clicking on controls, options, components, etc.); or events generated when a smart terminal plays the animation by vibration; or events generated when voice audio input for the animation is received, etc.

[0014] The multimedia data (or interactive animation) in this application embodiment may include at least one audio-visual material resource. That is, the interactive animation may be composed of at least one audio-visual material resource, and each audio-visual material resource may include one or more consecutive frames of images. When an audio-visual material resource includes one frame of images, it is an image; when it includes multiple consecutive frames of images, it can be understood as a short animation. During the playback of the interactive animation, if a certain audio-visual material resource included in the interactive animation requires an interactive event to be executed, then the audio-visual material resource is played only when an interactive event targeting that audio-visual material resource is detected. This interactive animation can improve user interactivity. It should be noted that the images mentioned in this application embodiment may refer to photos, video frames, or other images captured by various cameras, or animated images such as cartoons created using tools. Furthermore, the audio-visual material resources may also be audio resources, text resources, etc.

[0015] A schematic diagram of an exemplary process for playing an interactive animation can be found here. Figure 1 ,like Figure 1As shown, assuming the interactive animation includes two audio-visual resources, namely a first audio-visual resource and a second audio-visual resource, the second audio-visual resource is bound to an interactive event, meaning that playback of the second audio-visual resource is only triggered when an interactive event targeting the second audio-visual resource is detected; then the process of playing this interactive animation may include: First, when the interactive animation is started, the first audio-visual resource is triggered to play; if the first audio-visual resource finishes playing, a prompt message is output, prompting the user to perform an interactive operation to trigger playback of the second audio-visual resource. Second, when an interactive event is detected (such as an event generated when an interactive operation is performed), the second audio-visual resource is played. It should be noted that... Figure 1 This only provides a simple interactive animation process. In real-world applications, the playback process of the interactive animation can change; for example, if no interactive operation is detected within a preset time period (e.g., within 2 seconds) after the first audio-visual material resource finishes playing, the first audio-visual material resource can be played in a loop, or a prominent prompt message can be displayed to remind the user to perform an interactive operation; furthermore, the prominent prompt message mentioned above can also be an audio-visual material resource or a part of an audio-visual material resource, which can be in the form of text, components, options, etc.

[0016] Based on the above description of multimedia data, this application proposes a multimedia data processing scheme. This scheme supports configuring N audio-visual resources contained in multimedia data using an animation protocol to obtain configuration information for each audio-visual resource. This method of configuring audio-visual resources using an animation protocol enables the creation of multimedia data without code, improving the efficiency of multimedia data creation. Furthermore, the configuration information for the target resource includes interactive event information, enriching the playback effect of the created multimedia data. Simultaneously, users can create multimedia data through the animation protocol, allowing each user to create multimedia data according to their own scenario needs, improving the flexibility of multimedia data creation. In addition, the animation playback engine used for parsing multimedia data can be deployed on multi-terminal business platforms (such as business platforms corresponding to various operating systems), enabling the creation and parsing of multimedia data on these platforms, effectively promoting the application of interactive multimedia data.

[0017] The aforementioned multimedia data processing solution includes two parts: multimedia data creation and multimedia data parsing. The creation and parsing of multimedia data can be performed by the same or different computer devices, and the computer device used for parsing the multimedia data has an animation playback engine deployed based on an animation protocol. The animation playback engine can be deployed on the computer device or in an application running on the computer device. The application can include, but is not limited to: ① applications installed and running on the computer device; ② installation-free applications, i.e., applications that can be used without downloading and installation, commonly known as applets, which usually run as subroutines on the client; ③ web applications opened through a browser; etc. The computer devices here can include, but are not limited to: terminal devices such as smartphones, tablets, laptops, and desktop computers; or service devices such as data processing servers, web servers, and application servers. Terminal devices can be smartphones, tablets, laptops, desktop computers, smart speakers, smartwatches, etc., but are not limited to these. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server here can also be a node server on a blockchain. Terminal devices and service devices can be connected directly or indirectly through wired or wireless communication, which is not limited herein. For ease of explanation, the following description will use a computer device executing the multimedia data processing scheme proposed in the embodiments of this application as an example to illustrate the embodiments of this application.

[0018] Based on the multimedia data processing scheme described above, this application proposes a more detailed multimedia data processing method. The multimedia data processing method proposed in this application will be described in detail below with reference to the accompanying drawings.

[0019] Figure 2 The illustration shows a flowchart of a multimedia data processing method according to an exemplary embodiment of this application; the multimedia data processing method can be executed by a computer device, and the multimedia data processing method may include, but is not limited to, steps S201-S202: S201: Obtain the multimedia data to be played. The multimedia data includes N audio-visual material resources and configuration information obtained by configuring each audio-visual material resource based on the animation protocol, where N is an integer greater than 1. The multimedia data to be played has interactive characteristics; therefore, the configuration information of one or more audio-visual material resources included in the multimedia data contains interactive event information. This interactive event information is used to monitor interactive events related to the audio-visual material resources. For ease of explanation, the following description will use the configuration information of the target material resource among the N audio-visual material resources, including interactive event information, as an example.

[0020] In one implementation, assuming the multimedia data is an interactive animation, during the playback of the interactive animation, if an interaction event targeting the target material resource is detected, the target material resource is triggered to play; if no interaction event targeting the target material resource is detected, the target material resource is not triggered to play. Of course, if the multimedia data includes other audio-visual materials besides the target material resource, then after the target material resource finishes playing, the playback of these other audio-visual materials can also be triggered. For example, see... Figure 3a Assume the multimedia data includes three audio-visual resources, N=3, namely audio-visual resource 301, audio-visual resource 302, and audio-visual resource 303; wherein audio-visual resource 301 is video (or video stream), audio-visual resource 302 is image, and audio-visual resource 303 is video; if only audio-visual resource 302 has configuration information including interactive event information, then when playing multimedia data, audio-visual resource 3011 can be played automatically. Only when an interactive event for audio-visual resource 302 is detected according to the interactive event information of audio-visual resource 302 will the playback of audio-visual resource 302 be triggered, and audio-visual resource 303 will be played automatically after the playback of audio-visual resource 302 ends.

[0021] It is worth mentioning that, Figure 3a During the interactive animation playback shown, the interaction event for audio-visual material resource 302 is a wipe event. A wipe event can refer to an event generated when a user performs a wipe operation within a target area of ​​the interactive animation playback interface. For example... Figure 3aThe explanation assumes one erase event. This means that as long as one interaction event is detected, audio-visual material resource 302 is played, and audio-visual material resource 303 is played after audio-visual material resource 302 has finished playing. However, this embodiment also supports more than one erase event. That is, the multimedia data includes multiple audio-visual material resources 302, and each audio-visual material resource 302 is associated with one erase event. In this case, audio-visual material resource 303 is only triggered to play after all multiple audio-visual material resources 302 have finished playing. This method, which allows the user to perform multiple erase operations before playing audio-visual material resource 303, improves the realism of the interaction. A schematic diagram illustrating multimedia data including multiple audio-visual material resources 302 can be found [link to schematic diagram]. Figure 3b ,like Figure 3b As shown, when a user performs an erase operation, the audio-visual material resource 302 bound to the erase event generated by the erase operation can be played.

[0022] It should be noted that the specific details of the configuration information for audio-visual material resources can be found in the relevant descriptions in subsequent embodiments.

[0023] S202: Call the animation playback engine based on the animation protocol to parse and play the multimedia data.

[0024] Specifically, the implementation method for parsing and playing multimedia data may include: First, parsing the audio-visual material resources of the first layer indicated by layer information and the configuration information of the audio-visual material resources of the first layer; the layer information of any audio-visual material resource can be used to indicate the playback layer of that audio-visual material resource in the multimedia data. For example, if the layer information of the target material resource is 1, it indicates that the playback layer of the target material resource is the first layer, that is, when playing multimedia data, the target material resource is played first. Second, playing the audio-visual material resources of the first layer according to the configuration information of the audio-visual material resources of the first layer. Finally, after the audio-visual material resources of the first layer have finished playing, parsing the audio-visual material resources of the second layer indicated by layer information and the configuration information of the audio-visual material resources of the second layer; if the audio-visual material resources of the second layer are the target material resources, then playing the target material resources according to the configuration information of the target material resources. Specifically, the target material resources can be played according to the resource style information and interactive event information contained in the configuration information of the target material resources. The resource style information included in the configuration information of the target material resource is used to declare the style of the target material resource. For example, the resource style information of the target material resource is used to declare the display position, display size, and display shape of the target material resource on the display screen of the computer device. The interaction event information of the target material resource can be used to monitor interaction events targeting the target material resource (such as events that dynamically obtain external data in the background, or events that detect triggering events on the display screen).

[0025] It should be noted that the implementation methods for playing target material resources according to the resource style information and interaction event information included in the configuration information of the target material resource differ depending on the type of interaction events monitored for the target material resource's interaction event information. This application embodiment uses external data acquisition event information and interaction processing event information of the target material resource as examples to exemplify the implementation methods for playing target material resources according to the resource style information and interaction event information included in the configuration information of the target material resource. Wherein: 1) The interactive event information of the target material resources is the external data acquisition event information; the external data acquisition event information is used to monitor interactive events that acquire external data for the target material resources; among them, external data can refer to data that is not configured in the multimedia data. For example, if the multimedia data is an interactive animation, and if it is necessary to display the nickname (or avatar, etc.) of the account of the currently logged-in computer device (or the application with the animation playback engine deployed) on the page where the interactive animation is playing, then the external data refers to the nickname of the account.

[0026] In specific implementation, if the parsed configuration information of the target material resource includes interactive event information that is external data acquisition event information, then a target external data acquisition request is initiated and sent according to the data mapping relationship recorded in the external data acquisition event information. If target external data is received in response to the target external data acquisition request, it indicates that an interactive event for the target material resource has been detected, and then animation playback rendering is performed based on the target external data and the target material resource. The data mapping relationship recorded in the external data acquisition event information includes the mapping relationship between the data to be played and the external data information defined for the target material resource. Therefore, the implementation method for initiating a target external data acquisition request based on the data mapping relationship recorded in the external data acquisition event information may include: determining the external data information corresponding to the data to be played on the target material resource based on the mapping relationship between the data to be played and the external data information defined for the target material resource; after determining the external data information corresponding to the data to be played, generating an external data acquisition request based on the external data information and sending the external data acquisition request, which is used to request the acquisition of the target external data indicated by the external data information.

[0027] The following is in conjunction with the appendix Figure 4 When the interactive event information is obtained from external data, the implementation method of playing the target material resource according to the resource style information and interactive event information contained in the configuration information of the target material resource is illustrated by the following example. Figure 4As shown, assume that computer device 401 is equipped with an animation playback engine based on an animation protocol, and the multimedia data is an interactive animation. The interactive animation includes two audio-visual material resources, namely audio-visual material resource 4011 and audio-visual material resource 4012. The configuration information of audio-visual material resource 4012 includes interactive event information, which is external data acquisition event information. It is further assumed that the external data is stored in storage server 402.

[0028] In the scenario where an animation playback engine based on an animation protocol parses the multimedia data, the process of parsing the multimedia data may include: ① Obtaining the audio-visual material resource 1 of the first layer indicated by the layer information and the configuration information of the audio-visual material resource 1 of the first layer, and playing the audio-visual material resource of the first layer according to the configuration information of the audio-visual material resource of the first layer (such as the resource style information contained in the configuration information). ② After playing the audio-visual material resource 1 according to the configuration information of the audio-visual material resource 1 of the first layer, parsing the audio-visual material resource 2 of the second layer indicated by the layer information; assuming that the audio-visual material resource 2 is the target material resource, and the external data acquisition event information of the target material resource is used to monitor and acquire the user information (such as the user nickname or user avatar) of the current user who is logged into the computer device, then when the user information of the current user is acquired, it is determined that an interaction event for the audio-visual material resource 2 has been detected. Optionally, in the scenario of obtaining the current user's user information, the data mapping relationship recorded in the external data acquisition event information can include: bitmap: $AVATAR; wherein, during the configuration of the target material resource, macro definition pre-embedded protocol processing can be performed, such as defining user information (e.g., user avatar and / or user nickname, etc.) in the bitmap field of the external data acquisition event information contained in the target material resource, and registering external data information using the $AVATAR field in the animation playback engine; then, during the parsing process, based on the data mapping relationship bitmap: $AVATAR recorded in the external data acquisition event information of audio-visual material resource 2, the external data information $AVATAR corresponding to the bitmap of the data information to be played on the target material resource (e.g., audio-visual material resource 2) can be determined. ③ An external data acquisition request is generated based on the external data information $AVATAR, and the external data acquisition request is sent to the storage server 402 so that the storage server 402 responds to the external data acquisition request to query the target external data indicated by the external data information $AVATAR. ④ If the target external data returned by the storage server 402 is received, and the target external data includes user information (such as user nickname and / or user avatar), it indicates that an interaction event for the target material resource has been detected. Then, the animation playback rendering process is performed based on the target external data and the target material resource. The user information of the current user, such as user nickname 403 and user avatar 404, can be displayed in the playback interface of the rendering process.

[0029] 2) The interaction event information of the target material resource is interaction processing event information; interaction processing event information is used to monitor interaction events generated when there is an interactive operation on the target material resource; for example, the interaction events monitored by the interaction processing event information include: gesture events, voice and audio events, and vibration events, etc. For example, gesture events may include: events generated when there is a trigger operation on page elements (such as buttons, options, touchable areas, etc.) in the playback interface (such as an interface for playing multimedia data); voice and audio events may include: events generated when a voice input operation is received on the target material resource; vibration events may include: events generated when a vibration operation of a computer device is detected that vibrates (or shakes) the target material resource.

[0030] In the specific implementation, if the configuration information of the target material resource includes interactive event information, then interactive events are monitored according to the interactive event information. If an interactive event is detected, a time-based processing notification is generated based on the interactive event, and an event processing notification is output. This event processing notification is used to notify the execution of business processing based on the interactive event. Then, animation playback and rendering are performed based on the business processing result fed back by the response event processing notification and the target material resource. It is understandable that the generated event processing notification and the feedback business processing result will be different in different multimedia data playback scenarios.

[0031] A schematic diagram illustrating an exemplary event processing notification and the feedback of business processing results can be found here. Figure 5 ,like Figure 5 As shown, assuming the scenario of playing multimedia data is a coin-grabbing scenario, computer device 501 is equipped with an animation playback engine based on an animation protocol. During the playback of multimedia data, computer device 501 displays button 5012 on the playback interface 5011. When button 5012 is selected, it indicates that the user is participating in the coin-grabbing activity. An event processing notification can be generated based on the interaction event corresponding to this selection operation. This event processing notification is used to notify the user to allocate coins based on the selection operation. The event processing notification can be sent to processing server 502, which performs the coin allocation operation. Processing server 502 then returns the allocation result to computer device 501. The business processing result responding to the event processing notification may include: a coin-grabbing result notification message (such as...). Figure 5 The message shown is 503), the number of gold coins allocated to the user, etc.; then, the animation playback rendering process can be performed according to the business processing results and target material resources. The playback interface after rendering will display the coin grabbing result notification message or the number of gold coins allocated to the user, etc.

[0032] In this embodiment, an animation protocol is supported for configuring N audio-visual resources contained in multimedia data, obtaining configuration information for each audio-visual resource. This method of configuring audio-visual resources using an animation protocol enables the creation of multimedia data without code, improving the efficiency of multimedia data creation. Furthermore, the configuration information for the target resource includes interactive event information, enriching the playback effects of the created multimedia data. In addition, users can create multimedia data through the animation protocol, allowing each user to create multimedia data according to their own scenario needs, improving the flexibility of multimedia data creation.

[0033] Figure 6 The illustration shows a flowchart of a multimedia data processing method according to an exemplary embodiment of this application; the multimedia data processing method can be executed by a computer device, and the multimedia data processing method may include, but is not limited to, steps S601-S604: S601: Determine N audio-visual resources from the audio-visual resource set. The audio-visual resource set contains multiple candidate audio-visual resources, so that the designer (such as the user creating multimedia data) can select N audio-visual resources from the audio-visual resource set to compose the multimedia data.

[0034] In one implementation, determining N audio-visual material resources from a set of audio-visual material resources may include: displaying a resource configuration interface, which includes a material resource display area containing at least one candidate audio-visual material resource; selecting audio-visual material resources from the material display area based on a resource selection operation; and generating multimedia data to be played based on the selected audio-visual material resources. In other words, a resource display interface can be displayed on the screen of the computer device used by the designer. This interface includes a material resource display area, which can display all or part of the candidate audio-visual material resources from the set of audio-visual material resources; N audio-visual material resources are selected from the material display area based on a resource selection operation, and these N audio-visual material resources serve as the material resources for generating multimedia data. More specifically, the resource configuration interface also includes a resource combination area, which is used to display the selected N audio-visual material resources. When each candidate audio-visual material resource displayed in the material resource display area is displayed in the form of a component, this embodiment of the application supports the designer to drag the candidate component corresponding to the candidate audio-visual material resource from the material resource display area to the resource combination area. At this time, the candidate audio-visual material resource corresponding to the dragged candidate component is called an audio-visual material resource, which is used to form multimedia data.

[0035] The exemplary process of selecting audio-visual resources by dragging and dropping components, as described above, can be found in [reference needed]. Figure 7 ,like Figure 7As shown, when a designer needs to design multimedia data, they can open the resource display interface 701. This resource display interface can be the interface of any application running on a computer device (such as an instant messaging application). The resource display interface 701 includes a material resource display area 7011 and a resource combination area 7012. The material resource display area 7011 contains candidate components corresponding to all or part of the candidate audio-visual material resources in the audio-visual material resource set. When some of the candidate audio-visual material resources contained in the audio-visual material resource set are hidden, this embodiment supports sliding to display the hidden candidate audio-visual material resources. When the candidate component 70111 corresponding to candidate audio-visual material resource 1, the candidate component 70112 corresponding to candidate audio-visual material resource 2, and the candidate component 70113 corresponding to candidate audio-visual material resource 3 in the material resource display area 7011 are dragged into the resource combination area 7012, it indicates that the designer wants to design multimedia data based on candidate audio-visual material resource 1, candidate audio-visual material resource 2, and candidate audio-visual material resource 3. It should be noted that... Figure 7 This is merely an exemplary method for designers to determine audio-visual material resources. In actual application scenarios, the method for determining audio-visual material resources may vary, and this application embodiment does not limit the method for determining audio-visual material resources.

[0036] Furthermore, this application embodiment also supports splitting audio-visual material resources selected from the material resource display area. The resulting sub-audio-visual material resources can be used as one audio-visual material resource for generating multimedia data. Specifically, if a splitting operation on a target audio-visual material resource is detected in the resource combination area, at least two sub-audio-visual material resources obtained from the splitting are displayed in the resource combination area; each sub-audio-visual material resource is used as an audio-visual material resource for generating multimedia data to be played. For example, if any selected audio-visual material resource is split into three sub-audio-visual material resources, namely sub-audio-visual material resource 1, sub-audio-visual material resource 2, and sub-audio-visual material resource 3, then these three sub-audio-visual material resources are similar to the audio-visual material resources selected from the material resource display area, that is, the sub-audio-visual material resources also support configuration, such as configuring interactive event information for any sub-audio-visual material resource.

[0037] S602: Uses an animation protocol to describe the configuration information of N audio-visual material resources to obtain the multimedia data to be played.

[0038] The animation protocol mentioned in this application embodiment may include the XA protocol (xaction). The XA protocol is a DSL (Digital Subscriber Line) description protocol. DSL description protocols are efficient and concise domain-specific languages ​​that greatly reduce the difficulty of understanding and using compared to general-purpose languages, while significantly improving development efficiency. Using the XA protocol to describe the configuration information of audio-visual material resources can add interactive features to multimedia data without being limited to using code to achieve interactivity. Furthermore, using the XA protocol to create multimedia data allows dynamically distributed audio-visual material resources to directly support interactive effects, greatly optimizing the implementation method and creation cost of multimedia data (or interactive animations). Specifically, designers can create multimedia data using an XA animation editor, which can be deployed in applications (such as clients, web applications, etc.) or mini-programs running within applications. This allows designers to complete the creation of multimedia data within the application, reducing creation complexity and improving creation efficiency.

[0039] This application describes the playback, interaction, and feedback behaviors of multimedia data using the XA protocol, and recreates the entire playback and interaction process of multimedia data using an animation playback engine based on the XA protocol. The description language (or fields) used to configure audio-visual material resources in the XA protocol can be found in Field Description Table 1.

[0040] Table 1

[0041] As shown in Table 1, the XA protocol's field description table includes various fields for describing audio-visual material resources. These fields can be used to store different configuration information for audio-visual material resources. The field description table also includes an explanation of the meaning of the configuration information stored in each field, as well as the data type (or field type) of each field. For example, the `type` field can store configuration information indicating the type of audio-visual material resource; the `st` field can store configuration information indicating the style of audio-visual material resource; and for some audio-visual material resources with unique characteristics, the `exts` field can be used to store extended information about the audio-visual material resource, which meets the designer's needs for the diversity of audio-visual material resources; and so on.

[0042] In detail, the configuration information obtained by configuring each audio-visual material resource using the fields contained in the XA protocol shown in Table 1 may include any one or more of the following information: ① The configuration information includes layer information, which indicates the playback layer of audio-visual resources within the multimedia data. In other words, the layer information of the playback layer for any one of the N audio-visual resources contained in the multimedia data indicates the playback order of that resource when playing the multimedia data (or playing N audio-visual resources). The earlier a resource appears in the playback order, the earlier it is played when the multimedia data is played. Playback order can also be represented by playback levels; in this implementation, the lower the playback level of a resource, the later it is played when the multimedia data is played. Specifically, the id field included in the XA protocol can be used to configure the layer information of the audio-visual resources. For example, the multimedia data to be played includes three audio-visual resources, N=3, namely audio-visual resource 1, audio-visual resource 2, and audio-visual resource 3. The layer information of the playback layer of audio-visual resource 1 is id:1, the layer information of the playback layer of audio-visual resource 2 is sequence number id:3, and the layer information of the playback layer of audio-visual resource 3 is id:2. Then, it is determined that during the playback of multimedia data, audio-visual resource 1 will be played first, and audio-visual resource 3 will be played only after audio-visual resource 1 has finished playing; audio-visual resource 2 will be played only after audio-visual resource 3 has finished playing.

[0043] It is worth noting that, in addition to being represented by the numerical sequence number (e.g., the layer information of the audio-visual material resource is represented by the sequence number id:1), the layer information of the audio-visual material resource can also be represented by the letter sequence number, such as the layer information of audio-visual material resource 1 being represented by the letter sequence number id:A, the layer information of audio-visual material resource 2 being represented by the letter sequence number id:C, and the layer information of audio-visual material resource being represented by the letter sequence number id:B. In actual application scenarios, the form of configuring the layer information of audio-visual material resources using the id field can also change, and this application embodiment does not limit this.

[0044] ② The configuration information includes resource type information, which declares the type of audio-visual material resources; for example, the type of audio-visual material resources is image, MP4, etc. Specifically, the type of audio-visual material resources can be configured using the type field included in the XA protocol; for example, if the type field configured for any audio-visual material resource is "image", it means that the type of any audio-visual material resource is image; as another example, if the type field configured for any audio-visual material resource is "mp4", it means that the type of any audio-visual material resource is mp4.

[0045] ③ Configuration information includes resource style information, which declares the style of audio-visual resources. The style of audio-visual resources can be understood as the display style of audio-visual resources when displayed on a computer device. Specifically, configuring the resource style of each audio-visual resource (including sub-audio-visual resources) yields the resource style information for each resource. This resource style information may include one or more of the following: display position, display area, display size (e.g., length, width), and display color (e.g., background color). Specifically, the st field included in the XA protocol can be used to configure the style of audio-visual resources; for example, if the st field configured for any audio-visual resource is "h=400", it means that the height of that audio-visual resource is 400 centimeters (or millimeters, etc.); as another example, if the st field configured for any audio-visual resource is "y=500", it means that that audio-visual resource is displayed at a position with a vertical coordinate of 500 on a coordinate axis established with the display screen.

[0046] ④ Configuration information includes interaction event information, which is used to monitor interaction events with audio-visual materials. As mentioned above... Figure 2 The related content described in the illustrated embodiments includes interactive event information contained in any audio-visual material resource, which may include one or more of the following: external data acquisition event information and interactive processing event information. External data acquisition event information can be used to monitor interactive events related to acquiring external data for the audio-visual material resource; for example, if parsing multimedia data reveals that the audio-visual material resource contains a `rule` field, indicating that target external data needs to be acquired for rendering the audio-visual material resource, then an interactive event related to the audio-visual material resource is detected. Interactive processing event information can be used to monitor interactive events generated during interactive operations on the audio-visual material resource; for example, if a trigger operation on a target element in the playback interface is detected, then an interactive event is determined to trigger playback of the audio-visual material resource (such as the target material resource) bound to the interactive event information. Specifically, the `tr` field included in the XA protocol can be used to configure the interactive events of the audio-visual material resource.

[0047] ⑤ The configuration information includes data acquisition information, which is used to acquire dynamically mapped data. Dynamically mapped data can refer to data that can change; for example, depending on the account currently logged into the computer device, the acquired avatar, nickname, and other data corresponding to the account will be different. In this embodiment, the acquired data such as the avatar and nickname corresponding to the account that can change is called dynamically mapped data. Since the dynamically mapped data is acquired from the party registering the account (such as the application party), it can also be called external data. Dynamically mapped data can be data defined using macro definitions; macro definitions can be used to define data mapping relationships. For example, the mapping relationship between the data to be played and the external data information defined for the target material resource can include: bitmap: $AVATAR. In the XA protocol, the rule field can be used to store the data acquisition information configured for the audio-visual material resource; for example, if the rule field configured for any audio-visual material resource is "$AVATAR", it means that external data is acquired from the macro definition according to the $AVATAR field.

[0048] As shown in Table 1, in addition to supporting the configuration of the above-mentioned configuration information for audio-visual material resources, this application embodiment also supports the configuration of playback behavior for audio-visual material resources. Specifically, playback behavior is configured for each audio-visual material resource to obtain behavior rule information for each audio-visual material resource; wherein, the behavior rule information for audio-visual material resources includes one or more of rotation, translation, scaling, and transparency. For example, if the behavior rule information configured for any audio-visual material resource includes rotation, then when playing any audio-visual material resource, that audio-visual material resource can display a rotation animation effect.

[0049] It should be noted that: ① The configuration information for each audio-visual resource based on the XA protocol can include one or more of the configuration information described above; specifically, different configuration information can be configured for each audio-visual resource according to the designer's business needs; for example, the audio-visual resources to be configured include audio-visual resource 1, audio-visual resource 2, and audio-visual resource 3; if only audio-visual resource 2 is desired to have interactive features, the rule field can be used to configure event interaction information only for audio-visual resource 2. ② The above describes the configuration information of each audio-visual resource in multimedia data using the XA protocol, that is, the XA protocol can be used to implement interactive multimedia data; it is worth noting that the XA protocol can also implement view layout rendering; specifically, the XA protocol (or XA description protocol) contains technical architecture knowledge of layout rasterization and dynamic data binding mapping, therefore, the XA protocol can also be used for view rendering.

[0050] Below are some exemplary configuration information for audio-visual resources described using the XA protocol. For example, the code "id":3 can be used to describe the layer information of the playback layer of the audio-visual resource, that is, the playback layer of the audio-visual resource is layer 3; similarly, when the 3 in "id":3 changes to 4, it indicates that the playback layer of the audio-visual resource is layer 4. Another example is the code "type": "pagview" used to describe the resource type information of the audio-visual resource, such as this code declaring the type of the audio-visual resource as an image. Yet another example is the code "st":{"w":750,"h":400,"x":0,"y":500,"align":8} used to describe the resource style information of the audio-visual resource, such as this code snippet declaring the display position of the audio-visual resource on the display terminal, and the display style (e.g., aligned display); and so on. Based on the configuration information of the above audio-visual material resources, it can be known that: the playback layer of the audio-visual material resources is layer 3; the resource type of the audio-visual material resources is an image; and the display position of the audio-visual material resources on the display screen is 0 on the x-axis and 500 on the y-axis, the display size is 750 cm (or mm) wide and 400 cm (or mm) high, and the display style is an aligned style.

[0051] In summary, by using an animation protocol to configure the N audio-visual resources contained in the multimedia data, the configuration information of each audio-visual resource is obtained. This method of configuring audio-visual resources using an animation protocol enables the creation of multimedia data without code, improving the efficiency of multimedia data creation. Furthermore, users can create multimedia data through the animation protocol, which allows the created multimedia data to meet the actual scenario-based needs of users, and each user can become a creator of multimedia data.

[0052] S603: Obtain multimedia data to be played.

[0053] It should be noted that the specific implementation process shown in step S603 can be found in [reference needed]. Figure 2 The specific implementation process of step S202 in the illustrated embodiment will not be repeated here.

[0054] S604: Calls an animation playback engine based on an animation protocol to parse and play multimedia data.

[0055] This application embodiment uses an animation protocol to configure the various audio-visual material resources contained in the multimedia data. To facilitate the parsing of multimedia data configured using the animation protocol, this application embodiment constructs an animation playback engine based on the animation protocol. This animation playback engine can be used to parse multimedia data created using the animation protocol. An exemplary architecture diagram of the animation playback engine built based on the animation protocol can be found in [reference needed]. Figure 8 ,like Figure 8 As shown, the animation playback engine based on the animation protocol includes a business layer (or interactive event layer), a view layer (or view rendering layer), an animation layer, and a core layer (or engine core control layer); among which: ① The business layer includes records of the mapping relationship (or data mapping rule) between the data information to be played and external data information defined for the target material resources. The business layer can be used to manage data mapping relationships, such as registration and management (addition, deletion, or update). The business layer has interactive functions and can serve as a communication interface between the animation playback engine and external devices, used for communication processing with external devices and other layers of the animation playback engine. For example, the business layer can be used for monitoring interactive events. ② The view layer includes various types of view components, proxy view components, and drawable resources. The view layer can render the playback interface of the multimedia data based on its contained elements, that is, render the content of each part of the playback interface to realize the display of the multimedia data content. ③ The animation layer contains N audio-visual material resources contained in the multimedia data and the configuration information of each audio-visual material resource. When the multimedia data is an interactive animation, the animation layer contains interactive animations and animation control elements such as interpolators. ④ The core layer is primarily responsible for parsing multimedia data. It also manages audio-visual resources, such as interactive events, data mapping relationships, and interpolators. In summary, the business layer in the animation playback engine serves as an external interface, handling data communication between the engine and external devices. The view layer, animation layer, and core layer within the animation playback engine are mainly used for managing and parsing multimedia data.

[0056] The following is combined Figure 8 The diagram shows the architecture of the animation playback engine, and Figure 9 The flowchart shown here illustrates the process of parsing multimedia data by the animation playback engine. This serves as a summary of the multimedia data parsing process described in steps S201-S202. Figure 8 as well as Figure 9As shown, firstly, when it is necessary to parse the multimedia data to be played for animation playback rendering, the animation playback engine obtains the multimedia data to be played. The methods for obtaining the multimedia data to be played can include: the animation playback engine obtaining the multimedia data to be played sent by an external device through the business layer, or obtaining the multimedia data to be played from the animation playback engine's local storage space. Secondly, the core layer can perform style initialization of the audio-visual material resources based on the N audio-visual material resources of the multimedia data (such as initializing the length, width, and height of the audio-visual material resources), create a playback timeline (such as creating a playback timeline based on the layer information of each audio-visual material resource), and so on. Secondly, the animation layer and view layer initialize the playback of the first layer of audio-visual resources among the N audio-visual resources by parsing the results of each audio-visual resource. If, during the playback of multimedia data, the business layer detects an interactive event indicated by the interactive event information contained in the target material resource, it triggers the playback of the target material resource. After the target material resource finishes playing, it triggers the playback of the adjacent audio-visual resources among the N audio-visual resources whose playback layer is below or after the target material resource. Specifically, when the interactive event information is external data acquisition event information, the business layer performs external data escaping processing, and the animation playback engine sends an external data acquisition request through the business layer. When the interactive event information is interactive processing event information, the animation playback engine sends an event processing notification through the business layer. Finally, the animation layer obtains the response result of the interactive event from the business layer (such as target external data or business processing result), and performs dynamic playback rendering processing on the target material resource based on the response result and the target material resource to achieve playback of the target material resource. Additionally, as... Figure 9 As shown, this application embodiment also supports updating the data mapping relationship according to business needs. When the data mapping relationship is updated, the target external data obtained based on the data mapping relationship will change, and the style of the playback interface presented after the target material resources are rendered by animation based on the target external data and target material resources will also change.

[0057] Furthermore, combined Figure 10a and Figure 10bThe interactive events indicated by the interactive event information contained in the target material resources shown include events generated when performing a wipe operation in the multimedia data playback interface. Taking this as an example, the background execution flow and interface presentation flow of the animation playback engine parsing multimedia data are introduced. Specifically, after the animation playback engine obtains the multimedia data to be played, firstly, the core layer of the animation playback engine parses the multimedia data to obtain the parsing results. These parsing results include the parsing results of the audio-visual material resources of the first layer indicated by the layer information. Secondly, the animation layer plays the audio-visual material resources of the first layer based on the parsing results of the first layer's audio-visual material resources; a schematic diagram of the interface for playing the first layer's audio-visual material resources can be found in [reference needed]. Figure 10b The playback interface 1001 is shown. Next, after the audio-visual material resources in the first layer finish playing, the audio-visual material resources in the second layer, as indicated by the layer information, are parsed. If the audio-visual material resources in the second layer are the target material resources, it is monitored whether there are any interactive events indicated by the interactive event information contained in the target material resources. This interactive event information may include one or both of external data acquisition event information and interactive processing event information. Figure 10a Taking interactive event information as an example (such as events generated when performing a wipe operation), the system will then trigger the playback of the target material resource (such as...). Figure 10b The playback interface shown is 1002. Finally, after the target material resource finishes playing, if there are still audio-visual materials in the multimedia data with a playback level lower than the target material resource, then each audio-visual material resource will be played sequentially according to the playback timeline (e.g., ...). Figure 10b The playback interface shown in 1003 enables the playback of multimedia data.

[0058] In this embodiment, an animation protocol can be used to configure the N audio-visual resources contained in the multimedia data, obtaining the configuration information for each audio-visual resource. This method of configuring audio-visual resources using an animation protocol enables the creation of multimedia data without code, improving the efficiency of multimedia data creation. Furthermore, the configuration information for the target resource includes interactive event information, enriching the playback effects of the created multimedia data. Simultaneously, users can create multimedia data through the animation protocol, allowing each user to create multimedia data according to their own scenario needs, improving the flexibility of multimedia data creation. In addition, the animation playback engine used to parse the multimedia data can be deployed on multi-terminal business platforms (such as Android business platforms, iOS business platforms, and Web business platforms), realizing open-source creation and parsing of multimedia data and improving the usability of multimedia data.

[0059] Figure 11A schematic diagram of a multimedia data processing apparatus according to an embodiment of this application is shown; the multimedia data processing apparatus is disposed in a computer device. In one embodiment, the multimedia data processing apparatus can be used to perform... Figure 2 , Figure 6 In the illustrated embodiment, the multimedia data processing device can be a client running on a computer device (such as an application with an animation playback engine deployed); the specific implementation of the units included in the multimedia data processing device can be found in the description of the relevant content in the foregoing embodiments. Please refer to... Figure 11 The multimedia data processing apparatus of this application embodiment includes the following units: The acquisition unit 1101 is used to acquire multimedia data to be played. The multimedia data includes N audio-visual material resources and configuration information obtained by configuring each audio-visual material resource based on the animation protocol. The configuration information of the target material resource among the N audio-visual material resources includes interactive event information, where N is an integer greater than 1. Processing unit 1102 is used to call an animation playback engine based on the animation protocol to parse and play the multimedia data; During the parsing and playback process, if the target material resource is parsed, the target material resource is played according to the interaction event corresponding to the monitored interaction event information.

[0060] In one implementation, the configuration information obtained by configuring each audio-visual material resource based on the animation protocol includes any one or more of the following: This indicates the layer information of the playback layer in the multimedia data for audio-visual materials. Declare the type of audio-visual material resources; Declare the style of the audio-visual materials; Monitor interaction event information related to interactive events involving audio-visual materials resources; Obtain information about dynamically mapped data.

[0061] In one implementation, when the processing unit 1102 calls the animation playback engine based on the animation protocol to parse and play the multimedia data, it is specifically used for: Parse the audio-visual material resources of the first layer indicated by the layer information and the configuration information of the audio-visual material resources of the first layer; Play the audio-visual resources of the first layer according to the configuration information of the audio-visual resources of the first layer; After the audio-visual material resources of the first layer have finished playing, the audio-visual material resources of the second layer indicated by the layer information and the configuration information of the audio-visual material resources of the second layer are parsed. If the audio-visual material resource of the second layer is the target material resource, then the target material resource is played according to the resource style information and interactive event information contained in the configuration information of the target material resource.

[0062] In one implementation, when the processing unit 1102 plays the target material resource according to the resource style information and interaction event information included in the configuration information of the target material resource, it is specifically used for: If the interactive event information included in the configuration information of the target material resource is determined to be external data acquisition event information, then a target external data acquisition request is initiated based on the data mapping relationship recorded in the external data acquisition event information. If the target external data is received in response to the target external data acquisition request, then animation playback and rendering are performed based on the target external data and the target material resources.

[0063] In one implementation, the data mapping relationship of the external data acquisition event information record includes: the mapping relationship between the data information to be played and the external data information defined for the target material resource; When the processing unit 1102 initiates a target external data acquisition request based on the data mapping relationship of the external data acquisition event information record, it is specifically used for: Based on the mapping relationship between the data information to be played and the external data information defined for the target material resource, determine the external data information corresponding to the data information to be played on the target material resource; An external data acquisition request is generated based on the external data information, and the external data acquisition request is sent. The external data acquisition request is used to request the acquisition of the target external data indicated by the external data information.

[0064] In one implementation, the animation playback engine based on the animation protocol includes a business layer, in which the mapping relationship between the data information to be played and the external data information defined for the target material resource is recorded; When processing unit 1102 sends the external data acquisition request, it is specifically used for: The external data acquisition request is sent through the business layer.

[0065] In one implementation, when the processing unit 1102 plays the target material resource according to the resource style information and interaction event information included in the configuration information of the target material resource, it is specifically used for: If the configuration information of the target material resource is parsed to include interactive event information, then the interactive events are monitored according to the interactive event information. If the interaction event is detected, an event processing notification is generated based on the interaction event and output. The event processing notification is used to notify the execution of business processing based on the interaction event. Based on the business processing results fed back from the event handling notification and the target material resources, the animation playback and rendering process is performed.

[0066] In one implementation, the processing unit 1102 is configured such that the animation playback engine based on the animation protocol includes a business layer, and when outputting the event handling notification, it is specifically used for: The event handling notification is sent through the business layer.

[0067] In one implementation, the processing unit 1102 is further configured to: The resource configuration interface is displayed, which includes a material resource display area, and the material resource display area contains at least one candidate audio-visual material resource. Select audio-visual material resources from the resource material display area according to the resource selection operation; The multimedia data to be played is generated based on the selected audio-visual materials.

[0068] In one implementation, the resource configuration interface further includes a resource combination area, which is used to display the selected N audio-visual material resources; the processing unit 1102 is also used to: If a splitting operation targeting the target audio-visual material resource is detected within the resource combination area, at least two sub-audio-visual material resources resulting from the splitting are displayed within the resource combination area. Each sub-audio-visual material resource is used as an audio-visual material resource for generating the multimedia data to be played, and different layer information is configured for each.

[0069] In one implementation, the processing unit 1102 is further configured to: Configure resource styles for each audio-visual material resource to obtain resource style information for each audio-visual material resource; The resource style information of audio-visual materials includes one or more of the following: display position, display size, display area, and display color.

[0070] In one implementation, the processing unit 1102 is further configured to: Configure playback behavior for each audio-visual resource to obtain behavior rule information for each audio-visual resource. The behavioral rules information of audio-visual material resources include one or more of the following: rotation, translation, scaling, and transparency.

[0071] According to one embodiment of this application, Figure 11 The various units in the multimedia data processing method shown can be individually or entirely merged into one or more other units, or some of the units can be further divided into multiple functionally smaller units. This can achieve the same operation without affecting the technical effect of the embodiments of this application. The above units are based on logical function division. In practical applications, the function of one unit can also be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of this application, the multimedia data processing apparatus may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented by multiple units working together. According to another embodiment of this application, the multimedia data processing apparatus can be executed by running on a general-purpose computing device, such as a computer, which includes processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM). Figure 2 , Figure 6 The computer program (including program code) for each step involved in the corresponding method shown, to construct such... Figure 11 The multimedia data processing apparatus shown herein, and the multimedia data processing method for implementing the embodiments of this application, are described. A computer program may be recorded on, for example, a computer-readable recording medium, loaded onto the aforementioned computing device via the computer-readable recording medium, and executed therein.

[0072] In this embodiment, an animation protocol can be used to configure the N audio-visual resources contained in the multimedia data, obtaining the configuration information for each audio-visual resource. This enables code-free creation of multimedia data, improving the efficiency of multimedia data creation. Furthermore, the configuration information for the target resource includes interactive event information, enriching the playback effects of the created multimedia data. Additionally, users can create multimedia data through the animation protocol, allowing each user to create multimedia data according to their own scenario needs, thus improving the flexibility of multimedia data creation.

[0073] Figure 12 A schematic diagram of the structure of a computer device provided in an exemplary embodiment of this application is shown. Please refer to... Figure 12 The computer device includes a storage device 1201 and a processor 1202. In this embodiment, the computer device also includes a network interface 1203 and a user interface 1204. The computer device can be a smartphone, tablet, smart wearable device, etc., and can access the Internet through the network interface 1203 to communicate and exchange data with servers and other computer devices. The user interface 1204 can include a touch screen, etc., and can receive user operations and display various interfaces to the user to facilitate receiving user operations.

[0074] Storage device 1201 may include volatile memory, such as random-access memory (RAM); storage device 1201 may also include non-volatile memory, such as flash memory, solid-state drive (SSD), etc.; storage device 1201 may also include a combination of the above types of memory.

[0075] Processor 1202 may be a central processing unit (CPU). Processor 1202 may further include hardware chips. The aforementioned hardware chips may be application-specific integrated circuits (ASICs), programmable logic devices (PLDs), etc. The aforementioned PLDs may be field-programmable gate arrays (FPGAs), generic array logic (GALs), etc.

[0076] The storage device 1201 in this embodiment stores a computer program. The processor 1202 calls the computer program in the storage device. When the computer program is executed, the processor 1202 can be used to implement the above-mentioned functions, such as... Figure 2 , Figure 6 The methods described in the corresponding embodiments, etc.

[0077] In one embodiment, the computer device may correspond to a terminal device or a server; the storage device 1201 stores a computer program; the processor 1202 loads and executes the computer program to implement the corresponding steps in the above-described multimedia data processing method embodiment; specifically, the processor 1202 is used to perform the following steps: Obtain multimedia data to be played, the multimedia data including N audio-visual material resources and configuration information obtained by configuring each audio-visual material resource based on the animation protocol, the configuration information of the target material resource among the N audio-visual material resources includes interactive event information, and N is an integer greater than 1; The multimedia data is parsed and played by an animation playback engine based on the animation protocol. During the parsing and playback process, if the target material resource is parsed, the target material resource is played according to the interaction event corresponding to the monitored interaction event information.

[0078] In one implementation, the configuration information obtained by configuring each audio-visual material resource based on the animation protocol includes any one or more of the following: This indicates the layer information of the playback layer in the multimedia data for audio-visual materials. Declare the type of audio-visual material resources; Declare the style of the audio-visual materials; Monitor interaction event information related to interactive events involving audio-visual materials resources; Obtain information about dynamically mapped data.

[0079] In one implementation, when the processor 1202 invokes an animation playback engine based on the animation protocol to parse and play the multimedia data, it specifically performs the following steps: Parse the audio-visual material resources of the first layer indicated by the layer information and the configuration information of the audio-visual material resources of the first layer; Play the audio-visual resources of the first layer according to the configuration information of the audio-visual resources of the first layer; After the audio-visual material resources of the first layer have finished playing, the audio-visual material resources of the second layer indicated by the layer information and the configuration information of the audio-visual material resources of the second layer are parsed. If the audio-visual material resource of the second layer is the target material resource, then the target material resource is played according to the resource style information and interactive event information contained in the configuration information of the target material resource.

[0080] In one implementation, when the processor 1202 plays the target material resource according to the resource style information and interaction event information included in the configuration information of the target material resource, it specifically performs the following steps: If the configuration information of the target material resource is parsed to include interactive event information that is external data acquisition event information, then a target external data acquisition request is initiated based on the data mapping relationship recorded in the external data acquisition event information. If the target external data is received in response to the target external data acquisition request, then animation playback and rendering are performed based on the target external data and the target material resources.

[0081] In one implementation, the data mapping relationship of the external data acquisition event information record includes: the mapping relationship between the data information to be played and the external data information defined for the target material resource; When processor 1202 initiates a target external data acquisition request based on the data mapping relationship of the event information record obtained from the external data, it specifically performs the following steps: Based on the mapping relationship between the data information to be played and the external data information defined for the target material resource, determine the external data information corresponding to the data information to be played on the target material resource; An external data acquisition request is generated based on the external data information, and the external data acquisition request is sent. The external data acquisition request is used to request the acquisition of the target external data indicated by the external data information.

[0082] In one implementation, the animation playback engine based on the animation protocol includes a business layer, in which the mapping relationship between the data information to be played and the external data information defined for the target material resource is recorded; When the processor 1202 sends the external data acquisition request, it specifically performs the following steps: The external data acquisition request is sent through the business layer.

[0083] In one implementation, when the processor 1202 plays the target material resource according to the resource style information and interaction event information included in the configuration information of the target material resource, it specifically performs the following steps: If the configuration information of the target material resource is parsed to include interactive event information, then the interactive events are monitored according to the interactive event information. If the interaction event is detected, an event processing notification is generated based on the interaction event and output. The event processing notification is used to notify the execution of business processing based on the interaction event. Based on the business processing results fed back from the event handling notification and the target material resources, the animation playback and rendering process is performed.

[0084] In one implementation, the processor 1202 is configured to include a business layer in the animation playback engine based on the animation protocol, and when outputting the event handling notification, it specifically performs the following steps: The event handling notification is sent through the business layer.

[0085] In one implementation, the processor 1202 is further configured to perform the following steps: The resource configuration interface is displayed, which includes a material resource display area, and the material resource display area contains at least one candidate audio-visual material resource. Select audio-visual material resources from the resource material display area according to the resource selection operation; The multimedia data to be played is generated based on the selected audio-visual materials.

[0086] In one implementation, the resource configuration interface further includes a resource combination area, which is used to display the selected N audio-visual material resources; the processor 1202 is also used to perform the following steps: If a splitting operation targeting the target audio-visual material resource is detected within the resource combination area, at least two sub-audio-visual material resources resulting from the splitting are displayed within the resource combination area. Each sub-audio-visual material resource is used as an audio-visual material resource for generating the multimedia data to be played, and different layer information is configured for each.

[0087] In one implementation, the processor 1202 is further configured to perform the following steps: Configure resource styles for each audio-visual material resource to obtain resource style information for each audio-visual material resource; The resource style information of audio-visual materials includes one or more of the following: display position, display size, display area, and display color.

[0088] In one implementation, the processor 1202 is further configured to perform the following steps: Configure playback behavior for each audio-visual resource to obtain behavior rule information for each audio-visual resource. The behavioral rules information of audio-visual material resources include one or more of the following: rotation, translation, scaling, and transparency.

[0089] In this embodiment, an animation protocol can be used to configure the N audio-visual resources contained in the multimedia data, obtaining the configuration information for each audio-visual resource. This enables code-free creation of multimedia data, improving the efficiency of multimedia data creation. Furthermore, the configuration information for the target resource includes interactive event information, enriching the playback effects of the created multimedia data. Additionally, users can create multimedia data through the animation protocol, allowing each user to create multimedia data according to their own scenario needs, thus improving the flexibility of multimedia data creation.

[0090] This application embodiment also provides a computer-readable storage medium (Memory), which is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the processing system of the computer device. Furthermore, the storage space also stores computer programs (including program code) suitable for loading and execution by the processor 1202. It should be noted that the computer-readable storage medium here can be high-speed RAM memory or non-volatile memory, such as at least one disk storage device; optionally, it can also be at least one computer-readable storage medium located remotely from the aforementioned processor.

[0091] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the multimedia data processing methods provided in the various alternative embodiments described above.

[0092] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0093] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0094] The above description discloses only preferred embodiments of the present invention and should not be construed as limiting the scope of the present invention. Therefore, equivalent variations made in accordance with the claims of the present invention are still within the scope of the present invention.

Claims

1. A method for processing multimedia data, characterized in that, include: Acquire multimedia data to be played, the multimedia data including N audio-visual material resources and configuration information obtained by configuring each audio-visual material resource based on an animation protocol, the configuration information including at least layer information indicating the playback layer of each audio-visual material resource in the multimedia data, the configuration information of the target material resource among the N audio-visual material resources including interactive event information, the interactive event information including at least one of external data acquisition event information and interactive processing event information, where N is an integer greater than 1; The multimedia data is parsed and played by an animation playback engine based on the animation protocol. During the parsing and playback process, the following operations are performed layer by layer in the order indicated by the layer information: parsing the audio-visual material resources and their configuration information corresponding to the current playback layer, and playing the audio-visual material resources according to the configuration information; if the configuration information of the target material resource includes the external data acquisition event information, then the target material resource is subjected to animation playback rendering processing based on the target external data associated with the monitored corresponding interaction event; if the configuration information of the target material resource includes the interaction processing event information, then the target material resource is subjected to animation playback rendering processing based on the business processing result associated with the monitored corresponding interaction event.

2. The method as described in claim 1, characterized in that, The configuration information obtained by configuring each audio-visual material resource based on the animation protocol includes any one or more of the following: This indicates the layer information of the playback layer in the multimedia data for audio-visual materials. Declare the type of audio-visual material resources; Declare the style of the audio-visual materials; Monitor interaction event information related to interactive events involving audio-visual materials resources; Obtain information about dynamically mapped data.

3. The method as described in claim 2, characterized in that, The process of calling the animation playback engine based on the animation protocol to parse and play the multimedia data includes: Parse the audio-visual material resources of the first layer indicated by the layer information and the configuration information of the audio-visual material resources of the first layer; Play the audio-visual resources of the first layer according to the configuration information of the audio-visual resources of the first layer; After the audio-visual material resources of the first layer have finished playing, the audio-visual material resources of the second layer indicated by the layer information and the configuration information of the audio-visual material resources of the second layer are parsed. If the audio-visual material resource of the second layer is the target material resource, then the target material resource is played according to the resource style information and interactive event information contained in the configuration information of the target material resource.

4. The method as described in claim 3, characterized in that, Playing the target material resource according to the resource style information and interaction event information included in the configuration information of the target material resource includes: If the interactive event information included in the configuration information of the target material resource is determined to be external data acquisition event information, then a target external data acquisition request is initiated based on the data mapping relationship recorded in the external data acquisition event information. If the target external data is received in response to the target external data acquisition request, then animation playback and rendering are performed based on the target external data and the target material resources.

5. The method as described in claim 4, characterized in that, The data mapping relationship recorded in the external data acquisition event information includes: the mapping relationship between the data information to be played and the external data information defined for the target material resource; The step of initiating a target external data acquisition request based on the data mapping relationship of the event information record obtained from the external data includes: Based on the mapping relationship between the data information to be played and the external data information defined for the target material resource, determine the external data information corresponding to the data information to be played on the target material resource; An external data acquisition request is generated based on the external data information, and the external data acquisition request is sent. The external data acquisition request is used to request the acquisition of the target external data indicated by the external data information.

6. The method as described in claim 5, characterized in that, The animation playback engine based on the animation protocol includes a business layer, in which the mapping relationship between the data information to be played and the external data information defined for the target material resource is recorded; Sending the external data acquisition request includes: The external data acquisition request is sent through the business layer.

7. The method as described in claim 3, characterized in that, Playing the target material resource according to the resource style information and interaction event information included in the configuration information of the target material resource includes: If the configuration information of the target material resource is parsed to include interactive event information, then the interactive events are monitored according to the interactive event information. If the interaction event is detected, an event processing notification is generated based on the interaction event and output. The event processing notification is used to notify the execution of business processing based on the interaction event. Based on the business processing results fed back from the event handling notification and the target material resources, the animation playback and rendering process is performed.

8. The method as described in claim 7, characterized in that, The animation playback engine based on the animation protocol includes a business layer, and the output of the event handling notification includes: The event handling notification is sent through the business layer.

9. The method as described in claim 1, characterized in that, Before acquiring the multimedia data to be played, the process also includes: The resource configuration interface is displayed, which includes a material resource display area, and the material resource display area contains at least one candidate audio-visual material resource. Select audio-visual material resources from the resource material display area according to the resource selection operation; The multimedia data to be played is generated based on the selected audio-visual materials.

10. The method as described in claim 9, characterized in that, The resource configuration interface also includes a resource combination area, which is used to display the selected N audio-visual material resources; before generating the multimedia data to be played based on the selected audio-visual material resources, the interface further includes: If a splitting operation targeting the target audio-visual material resource is detected within the resource combination area, at least two sub-audio-visual material resources resulting from the splitting are displayed within the resource combination area. Each sub-audio-visual material resource is used as an audio-visual material resource for generating the multimedia data to be played, and different layer information is configured for each.

11. The method as described in claim 9 or 10, characterized in that, Before generating the multimedia data to be played based on the selected audio-visual material resources, the method further includes: Configure resource styles for each audio-visual material resource to obtain resource style information for each audio-visual material resource; The resource style information of audio-visual materials includes one or more of the following: display position, display size, display area, and display color.

12. The method as described in claim 9 or 10, characterized in that, Before generating the multimedia data to be played based on the selected audio-visual material resources, the method further includes: Configure playback behavior for each audio-visual resource to obtain behavior rule information for each audio-visual resource. The behavioral rules information of audio-visual material resources include one or more of the following: rotation, translation, scaling, and transparency.

13. A multimedia data processing apparatus, characterized in that, include: An acquisition unit is used to acquire multimedia data to be played. The multimedia data includes N audio-visual material resources and configuration information obtained by configuring each audio-visual material resource based on an animation protocol. The configuration information includes at least layer information indicating the playback layer of each audio-visual material resource in the multimedia data. The configuration information of the target material resource among the N audio-visual material resources includes interactive event information. The interactive event information includes at least one of external data acquisition event information and interactive processing event information, where N is an integer greater than 1. The processing unit is used to call the animation playback engine based on the animation protocol to parse and play the multimedia data; During the parsing and playback process, the following operations are performed layer by layer in the order indicated by the layer information: parsing the audio-visual material resources and their configuration information corresponding to the current playback layer, and playing the audio-visual material resources according to the configuration information; if the configuration information of the target material resource includes the external data acquisition event information, then the target material resource is subjected to animation playback rendering processing based on the target external data associated with the monitored corresponding interaction event; if the configuration information of the target material resource includes the interaction processing event information, then the target material resource is subjected to animation playback rendering processing based on the business processing result associated with the monitored corresponding interaction event.

14. A computer device, characterized in that, include: Storage device for storing computer applications; The processor executes a computer application stored in the storage device to implement the multimedia data processing method as described in any one of claims 1-12.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer application program, which, when executed, implements the multimedia data processing method as described in any one of claims 1-12.

Images