Media resource rendering method and apparatus, computer equipment and storage medium

Abstract

This disclosure provides a method and apparatus for rendering media resources, computer equipment, and storage media. The method includes: acquiring a plurality of media rendering resources in a rendering template in a media editing interface; generating first media data based on the plurality of media rendering resources, wherein the first media data includes a first rendering resource set that includes at least some of the media rendering resources; playing back the first media data in a media editing interface; and, in the process of playing back the first media data, adding in real time the playback effect of a second rendering resource set that includes other media rendering resources from the plurality of media rendering resources other than the first rendering resource set.

Description

Technical Field

[0004] , , , ,

[0005] , ,

[0001] [Cross - reference to Related Applications] This application is based on and claims priority to Chinese Patent Application No. 202310582866.7 filed on May 22, 2023, the disclosure of which is incorporated herein by reference in its entirety.

[0002] [Technical Field] This disclosure relates to the field of computer technology, specifically to a method and apparatus for rendering media resources, a computer device, and a storage medium.

Background Art

[0003] When rendering audio - video data including complex effects, generally a corresponding rendering template is set. The rendering template includes effects, filters, stickers, transitions, music, videos, etc., and each effect corresponds to a corresponding resource pack, but the resource pack is stored on a remote server and needs to be downloaded for use.

Summary of the Invention

[0004] Embodiments of this disclosure provide a method and apparatus for rendering media resources, a computer device, and a storage medium.

[0005] According to one aspect of the present disclosure, an embodiment of the present disclosure provides a method for rendering media resources, the method comprising: obtaining a plurality of media rendering resources in a rendering template in a media editing interface; generating first media data based on the plurality of media rendering resources, wherein the first media data includes a first rendering resource set that includes at least some of the media rendering resources; playing the first media data in a media editing interface; and, in the process of playing the first media data, adding in real time the playback effect of a second rendering resource set that includes other media rendering resources from the plurality of media rendering resources other than the first rendering resource set.

[0006] In another aspect of the present disclosure, an embodiment of the present disclosure provides a media resource rendering apparatus, the apparatus comprising: a resource acquisition module for acquiring a plurality of media rendering resources in a rendering template in a media editing interface; a generation module for generating first media data based on the plurality of media rendering resources, wherein the first media data comprises a first rendering resource set comprising at least some of the media rendering resources; a playback module for playing the first media data in a media editing interface; and an addition module for adding, in real time, the playback effect of a second rendering resource set comprising other media rendering resources from the plurality of media rendering resources other than the first rendering resource set, during the process of playing the first media data.

[0007] According to another aspect of the present disclosure, an embodiment of the present disclosure provides a computer device comprising memory and a processor, the memory and the processor being connected to each other in communication, the memory storing computer instructions, and the processor executing a method for rendering media resources of the first described embodiment or any one of the corresponding embodiments by executing the computer instructions.

[0008] According to another aspect of the present disclosure, embodiments of the present disclosure provide a computer-readable storage medium on which computer instructions are stored, and the computer instructions are used to cause a computer to perform a method for rendering a media resource of the first described embodiment or any one of the corresponding embodiments.

[0009] To more clearly illustrate specific embodiments of this disclosure or technical concepts in the prior art, the following is a brief introduction to the drawings that may be used in describing specific embodiments or the prior art. Clearly, the drawings in the following description are some embodiments of this disclosure, and those skilled in the art can obtain further drawings based on these without any creative effort. [Brief explanation of the drawing]

[0010] [Figure 1] This is a flowchart illustrating a method for rendering media resources according to some embodiments of this disclosure. [Figure 2] This is a flowchart of a method for rendering another media resource according to some embodiments of the present disclosure. [Figure 3] This is a flowchart of yet another media resource rendering method according to some embodiments of the present disclosure. [Figure 4] This is a schematic diagram of experimental results of media resource rendering methods according to some embodiments of this disclosure. [Figure 5] This is a structural block diagram of a media resource rendering device according to an embodiment of the present disclosure. [Figure 6] This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present disclosure. [Modes for carrying out the invention]

[0011] To further clarify the purpose, technical proposal, and advantages of the embodiments of this disclosure, the technical proposal of the embodiments of this disclosure will be described clearly and completely below, with reference to the drawings of the embodiments. Of course, the embodiments described are only a part of the embodiments of this disclosure, not all of them. All other embodiments obtained by a person skilled in the art without any creative effort based on the embodiments of this disclosure are all within the scope of protection of this disclosure.

[0012] When rendering audio-video data that includes complex effects such as effects, filters, stickers, transitions, music, and videos, the rendering template needs to include resource packs for these effects, filters, stickers, transitions, music, and videos. These resource packs typically reside on a remote server and need to be downloaded and used. Therefore, when rendering a single audio-video data file with complex effects and whose effect resource packs reside on a remote server, the rendering template must define descriptions of the remote resource packs, such as HTTP links. The rendering template then traverses all relevant remote resources, downloads them sequentially according to the playback time order, updates the local file paths in the rendering template after the download is complete, and reproduces the media effects based on the rendering template to obtain the audio-video effects.

[0013] Because rendering templates can involve a very large number of resources, if the above approach is adopted, and it is necessary to play back a preview effect at a certain point in time, it will be necessary to wait until all resources corresponding to that point in time have finished downloading before the media effect can be reproduced. This requires a very long loading time, and the loading time will be longer the worse the network conditions are, significantly impacting the user experience. At the same time, it is difficult to avoid resource download failures due to weak network signals, which will prevent the recovery of the media effects contained in the rendering template.

[0014] However, this disclosure considers that when restoring media effects corresponding to a rendering template, it is not necessary to render all media effects at once, and that some media effects, such as some effects, stickers, and filter effects, can be restored incrementally according to the playback time. For example, if a rendering template contains many sticker resources, the video may be rendered first, and then the stickers may be downloaded while it is playing, and rendering may be performed after each sticker has finished downloading. This gives the user the feeling that the effects in the template are being dynamically added one after another during playback, significantly reducing the loading time of the rendering template without affecting the user's media effect preview experience.

[0015] According to this, the proposed technology divides rendering resources into essential and non-essential categories, prioritizes the processing of essential rendering resources, and presents the corresponding media interface to the user, thereby avoiding long waiting times for the user. For non-essential rendering resources, rendering can be done while downloading based on the requirements at the time of playback, enhancing the user's preview experience of media effects without affecting the presentation of media effects.

[0016] Embodiments of the present disclosure provide embodiments of media resource rendering methods, and it should be noted that the steps shown in the flowcharts of the drawings can be performed on a computer system such as a set of computer-executable instructions, and although the flowcharts show a logical order, in some cases the steps shown or described may be performed in an order different from the order herein.

[0017] In this embodiment, a method for rendering media resources that can be used in the above-mentioned computer devices, such as mobile phones, tablet PCs, and computers, is provided. Figure 1 is a flowchart of the method for rendering media resources according to an embodiment of this disclosure, and as shown in Figure 1, the flow includes the following steps.

[0018] Step S101: In the media editing interface, retrieve multiple media rendering resources in the rendering template.

[0019] The media editing interface is used to configure the rendering template. The rendering template is used to define a description for the media rendering resource, to lay out the media rendering resource, and to load media data based on the rendering template. The media rendering resource is a resource pack necessary for generating media effects, and specifically, the media rendering resource may include video resource packs, audio resource packs, sticker resource packs, filter resource packs, effect resource packs, video resource packs, transition resource packs, etc. Here, the types of media rendering resources are not limited, and those skilled in the art can select them according to their actual needs.

[0020] The technician can pre-configure the rendering template in the media editing interface and configure the necessary media rendering resources for the rendering template. Accordingly, the media editing interface can obtain a plurality of media rendering resources included in the rendering template by analyzing the configuration information of the rendering template.

[0021] Step S102: Generate first media data based on a plurality of media rendering resources.

[0022] Here, the first media data includes a first rendering resource set, and the first rendering resource set includes at least some of the media rendering resources.

[0023] Input some of the media rendering resources included in the first rendering resource set into the rendering layer. The rendering layer can render some of the media rendering resources according to the layout related to the rendering template and generate the corresponding first media data.

[0024] Step S103: Play the first media data in the media editing interface.

[0025] Present the first media data in the media editing interface so that the user can preview the first media data related to the first rendering resource set.

[0026] Taking the first rendering resource set as an example, after identifying the video resource in the rendering template, the video resource is downloaded, the downloaded video resource is sent directly to the rendering layer, video rendering is performed, the video data corresponding to the video resource is presented in the media editing interface, and the user can view the video playback screen by playing the video data in the media editing interface. To understand this, the video resources included in the first rendering resource set may be distributed at any time node or time period of the video data, that is, the determination of the first rendering resource set may be independent of the time position and time order of its playback.

[0027] In step S104, during the process of playing the first media data, the playback effects of the second rendering resource set are added in real time. Here, the second rendering resource set includes other media rendering resources from among multiple media rendering resources, excluding those in the first rendering resource set.

[0028] The second rendering resource set is a collection of media rendering resources required for the playback of the first media data. Specifically, the second rendering resource set includes media rendering resources other than those included in the first rendering resource set from among multiple media rendering resources. The other media rendering resources included in the second rendering resource set may be determined based on the playback time of the first media data.

[0029] Specifically, in the process of playing the first media data, media rendering resources included in the second rendering resource set are sent to the rendering layer, rendering is performed, and playback effects related to the second rendering resource set are obtained. As a result, playback effects corresponding to the second rendering resource set can be added in real time after the first media data has been played.

[0030] As an example, if the first rendering resource set contains video resources and the second rendering resource set contains sticker resources, when rendering for the video resources is completed and the video playback screen is presented, the sticker resources in the rendering template are downloaded, rendering is performed on the sticker resources in the rendering layer, and sticker data can be obtained. The sticker data and video data are merged to dynamically add sticker effects to the video playback screen.

[0031] The media resource rendering method according to this embodiment generates and plays first media data using some of the multiple media rendering resources acquired in the media editing interface. After playing the first media data, processing is performed on other media rendering resources, and during the process of playing the first media data, the corresponding playback effects are added to the other media rendering resources in real time. As a result, the media resources in the rendering template do not need to wait until all of them have finished downloading during rendering, and can be downloaded and rendered as needed, thus reducing the loading time of media resources. Furthermore, during the rendering process, some media rendering resources can be processed preferentially to show the corresponding media data to the user, reducing the user's waiting time. Other media rendering resources can be played while rendering, thereby dynamically adding the corresponding playback effects after playing the first media data, reducing the impact on the presentation of media effects, and enhancing the user's media effect preview experience.

[0032] In this embodiment, a method for rendering media resources that can be used in the above-mentioned computer devices, such as mobile phones, tablet PCs, and computers, is provided. Figure 2 is a flowchart of the media resource rendering method according to an embodiment of this disclosure, and as shown in Figure 2, the flow includes the following steps.

[0033] In step S201, multiple media rendering resources are obtained in the rendering template within the media editing interface. For a detailed explanation, please refer to the relevant description corresponding to the above embodiment; no further explanation is provided here.

[0034] Step S202: Generate first media data based on multiple media rendering resources.

[0035] Here, the first media data includes a first rendering resource set, the first rendering resource set includes at least some media rendering resources.

[0036] Specifically, the first rendering resource set includes essential rendering resources from among multiple media rendering resources, and accordingly, step S202 above may also include the following:

[0037] Step S2021: Identify at least one required rendering resource from multiple media rendering resources.

[0038] Essential rendering resources are the initial playback resources required for rendering media effects, i.e., resources on which media effects are preferentially played. For example, if initial playback video is required, the essential rendering resource is the video resource; if initial playback audio is required, the essential rendering resource is the audio resource.

[0039] Multiple media rendering resources include both essential and non-essential rendering resources. Pre-configured rules identify the essential rendering resources from among the multiple media rendering resources, and these pre-configured rules are for identifying essential rendering resources.

[0040] In some embodiments, step S2021 may include the following:

[0041] Step a1: Identify whether each of the multiple media rendering resources has the required identifiers.

[0042] Required identifiers are identifier information carried by required rendering resources. When media rendering resources in a rendering template are retrieved, the identifier information carried by each media rendering resource is examined to determine whether the identifier information carried there is a required identifier or not.

[0043] Step a2: Determine one or more media rendering resources that carry the required identifiers among multiple media rendering resources as the required rendering resources.

[0044] In response to the determination that there is one media rendering resource carrying a required identifier in the rendering template, that media rendering resource, for example, all video clips on the main track, is determined as a required rendering resource. On the other hand, in response to the determination that there are multiple media rendering resources carrying a required identifier in the rendering template, those multiple media rendering resources, for example, only the first video on the main track, all clips on the main track, audio tracks, sticker tracks, etc., are determined as required rendering resources. At the same time, resources other than the required rendering resources are determined as non-required rendering resources.

[0045] This allows for the rapid identification and priority loading of essential rendering resources by identifying the required identifiers carried by media rendering resources, thereby ensuring the loading speed of essential rendering resources.

[0046] Step S2022: Process at least one required rendering resource to generate the first media data.

[0047] Here, the first media data includes at least one essential rendering resource.

[0048] The rendering layer can input the required rendering resources, perform rendering on those resources, and generate first media data containing those resources. For example, if the required rendering resources are video resources, the rendering layer can perform rendering on those resources and generate a video screen corresponding to those resources, allowing the user to preview the first media data associated with the required rendering resources through the video screen. To understand this, the required rendering resources may be distributed at any time node or time period of the video data; that is, the determination of the required rendering resources may be independent of their time position and time order during playback.

[0049] In step S203, the first media data is played back in the media editing interface. For a detailed explanation, please refer to the relevant description corresponding to the above embodiment, and no further explanation is provided here.

[0050] In step S204, during the process of playing the first media data, the playback effects of the second rendering resource set are added in real time. Here, the second rendering resource set includes media rendering resources other than the first rendering resource set among multiple media rendering resources. A detailed explanation is provided in the relevant description corresponding to the above embodiment and is not explained further here.

[0051] In some embodiments, step 104 or step S204 may include generating second media data based on the first media data and the second rendering resource set.

[0052] In some embodiments, the second media data includes the first media data and at least some media rendering resources in a second set of rendering resources that are added in real time.

[0053] The first media data is played, then the media rendering resources from the second rendering resource set are input to the rendering layer, and the rendering process is performed to fuse the first media data with the media data obtained from rendering the second rendering resource set, thereby generating the second media data. When the second media data is presented, the user can preview the playback effects of the first media data and other media rendering resources that are added to the first media data in real time.

[0054] In some embodiments, the second rendering resource set includes non-essential rendering resources from among multiple media rendering resources, and accordingly, the step of generating the second media data based on the first media data and the second rendering resource set described above is: Step B may include processing one or more non-essential rendering resources within a predetermined time period after processing essential rendering resources to obtain processing results, and then generating and playing back second media data that includes the non-essential rendering resources based on the processing results and first media data.

[0055] The pre-set time interval is the pre-set interval after rendering has been performed on the required rendering resources. In other words, rendering is performed on the non-essential rendering resources of the second set of rendering resources only after the rendering process for the required rendering resources has been completed. Specifically, the non-essential rendering resources are downloaded, the downloaded non-essential rendering resources are sent to the rendering layer, the rendering process is performed, and media data containing the non-essential rendering resources is obtained. Subsequently, the media data corresponding to the second set of rendering resources and the first set of media data are merged, the first set of media data is played, and then the second set of media data is played.

[0056] In this embodiment, by dividing media resources into essential rendering resources and non-essential rendering resources, when rendering is performed, rendering is prioritized for essential rendering resources to generate the corresponding first media data, which is then played back in the media editing interface. Further processing is performed on the non-essential rendering resources to generate the corresponding second media data, which is then played back based on the first media data. As a result, when rendering media resources, essential rendering resources are processed preferentially, the corresponding media editing interface is shown to the user, and the user's waiting time can be reduced. Non-essential rendering resources can be played back while rendering is in progress, thereby playing back the first media data and then dynamically adding media data corresponding to the second set of rendering resources in real time, reducing the impact on the presentation of media effects and enhancing the user's preview experience of media effects.

[0057] In some embodiments, step B may include the following:

[0058] Step b1: If the first media data is to be played for a predetermined duration, the playback position of the first media data is determined.

[0059] The pre-set time length is the pre-set playback time length of the first media data, and the playback position is the position reached after the first media data has been played for the pre-set time length. Specifically, the first media data is played according to the corresponding playback track, and the playback track sets the playback time of the media data according to the time dimension, meaning there is a one-to-one correspondence between the playback time and the media data. Based on the pre-set time length of the first media data, the position of the first media data on the playback track can be determined, and this position is the playback position of the first media data.

[0060] Taking the first media data as video data as an example, the video data is played according to the video track, and each frame of video corresponds to the start time and end time. After the video data has been played for a predetermined duration, the current playback position of the video data can be determined according to the predetermined duration, the start time of each frame of video, and the end time of each frame of video.

[0061] Step b2: Based on the playback position, determine the target resources required for the next playback position from the non-essential rendering resources.

[0062] After playing the first media data, each non-essential rendering resource can be downloaded based on a preset start and end time and loaded into the current first media data. To ensure loading continuity and demonstrate the effects of dynamically added resources, the playback process of the first media data can detect in real time whether or not a non-essential rendering resource needs to be loaded at the next playback position.

[0063] By downloading non-essential rendering resources that need to be loaded at the next playback position as target resources, even in poor network conditions, this method allows for the download of non-essential rendering resources according to playback demand, eliminating the need to wait for all non-essential rendering resources to be downloaded. This reduces resource loading waiting time and ensures media playback performance.

[0064] In some optional embodiments, step b2 may include the following:

[0065] Step b21: Obtain the load time for each non-essential rendering resource, and based on the load time, determine the target playback position for each non-essential rendering resource, based on the first media data.

[0066] Step b22: Compare the target playback position with the next playback position, and based on the comparison result, screen the non-essential rendering resources for the target resources required for the next playback position.

[0067] The load time represents the presentation time for each non-essential rendering resource, which is pre-configured according to actual demand, and without specifically limiting the load time, a person skilled in the art can set the load time for each non-essential rendering resource on the first media data based on debugging results or empirical values.

[0068] Based on the playback time of the first media data, the playback position of each playback time of the first media data on the playback track is determined. Based on the load time of non-essential rendering resources, the target playback position of the load time on the playback track can be determined.

[0069] If the next playback position of the first media data is the same as the target playback position of a non-essential rendering resource, this indicates that the corresponding non-essential rendering resource needs to be loaded at the next playback position, and the non-essential rendering resource is confirmed as the target resource required for the next playback position.

[0070] In this embodiment, by determining the load time corresponding to non-essential rendering resources, the target playback position based on the first media data can be determined, and the necessary target resources can be determined in accordance with the target playback position and the next playback position of the first media data, thereby enabling dynamic scheduling of non-essential rendering resources and facilitating the acquisition of the corresponding target resources based on the actual playback state.

[0071] Step b3: Process the target resources according to the pre-set priority and generate second media data corresponding to the target resources.

[0072] The pre-configured priority is the pre-configured download order for each target resource, and the second media data is the media data generated based on the rendering of the target resource. Specifically, each target resource is downloaded according to the pre-configured priority, and as the download of one target resource is completed, one target resource can be rendered. In other words, by performing a download and rendering operation on the target resources, the second media data associated with each target resource can be obtained sequentially.

[0073] In some optional embodiments, step b3 may include the following:

[0074] Step b31: Build multiple download tasks corresponding to the target resource.

[0075] Step b32: Prioritize multiple download tasks based on pre-set priorities and obtain the priority ordering result.

[0076] Step b33: The multiple download tasks are triggered sequentially according to the priority ordering result, and the second media data is generated.

[0077] Multiple download tasks are created for each target resource, each download task is added to a task queue, and each download task in the task queue is prioritized based on a pre-configured priority to obtain the priority prioritization result. During download, the download tasks are triggered according to this priority prioritization result. Specifically, different types of target resources are assigned different priorities, and each type of target resource is arranged in descending order of priority; that is, the higher the priority of a target resource, the more likely the corresponding download task is to be triggered.

[0078] For example, the priority for video resources might be 100, audio resources 99, filter resources 98, effect resources 97, and sticker resources 96. Different types of target resources correspond to one or more download tasks, and each download task is ordered according to the priority of the various types of target resources to obtain the priority ordering result for each download task. In this example, video resources have the highest priority, and download tasks corresponding to video resources are triggered preferentially.

[0079] When a download task is completed, it indicates that the resource pack corresponding to that download task has already been downloaded locally. At this point, rendering is performed on the downloaded resource pack to obtain the media data associated with that resource pack.

[0080] In this embodiment, by sequentially triggering each download task according to its priority ordering result, priority loading of required target resources is ensured, thereby enhancing the rendering efficiency of media resources.

[0081] In some optional embodiments, step b33 may include the following:

[0082] Step c1 detects whether the number of download tasks exceeds a predetermined limit for concurrent execution.

[0083] Step c2: In response to the number of download tasks exceeding a preset concurrent execution limit, one or more current download tasks are selected according to the preset concurrent execution limit.

[0084] Step c3: Trigger one or more current download tasks in order according to the priority ordering result to generate the second media data.

[0085] The pre-configured concurrent execution amount is the maximum number of concurrent tasks pre-configured in the service layer, for example, 3, 4, or 5. However, the pre-configured concurrent execution amount is not limited, and a person skilled in the art may determine it according to actual demand.

[0086] The number of download tasks corresponding to the target resource is statistically analyzed, and the number of download tasks is compared with a pre-configured concurrent execution limit to determine whether the number of download tasks corresponding to the target resource exceeds the pre-configured concurrent execution limit. If the number of download tasks exceeds the pre-configured concurrent execution limit, multiple download tasks with higher priority are selected from the multiple download tasks based on the pre-configured concurrent execution limit and priority ordering results, these download tasks are executed, the corresponding resource download process is triggered, rendering is performed on the downloaded resources, and the corresponding media data is obtained.

[0087] Specifically, by completing each download task, the highest-priority download task is continuously retrieved from the task queue to meet the maximum concurrent task load, and resource downloads continue, thereby enabling incremental rendering of the target resource.

[0088] To explain, following the playback process of the first media data, non-essential rendering resources required for the next playback position are detected in real time based on the playback time and position of the first media data, non-essential rendering resources related to the content to be played are downloaded preferentially, the download tasks corresponding to the non-essential rendering resources are reordered in terms of priority, and each download task is triggered in order according to the priority reordering result after the reordering.

[0089] In this embodiment, if the number of download tasks corresponding to the target resource exceeds a preset concurrent execution amount, scheduling is performed for multiple download tasks corresponding to the target resource according to a preset priority and a preset concurrent execution amount. After each download task is completed, the system schedules the download task with the highest priority from the remaining download tasks, thereby enabling incremental rendering of the target resource and significantly reducing resource loading time without affecting the user's preview of media effects.

[0090] Step b4: Play the second media data based on the first media data.

[0091] By playing the first media data using the media editing interface, and then loading the second media data obtained from rendering the target resource onto the current media editing interface, the second media data corresponding to each target resource can be dynamically added sequentially to the first media data and played back, thereby achieving dynamic loading and playback of the second media data.

[0092] As an example, if the first media data is video data and the second media data is audio data and sticker data, the playback times of the audio resource and sticker resource are set in advance. For example, the audio data and sticker data are added 2 seconds after the start of video data playback. When it is detected that the video data has been played up to 1.9 seconds, the audio resource and sticker resource required for the next playback position (i.e., the position corresponding to 2 seconds) are determined from the non-essential rendering resources, and the priority of the audio resource is higher than that of the sticker resource. Here, the audio resource and sticker resource are targeted resources, and the download is performed in the order that the audio resource takes priority, followed by the sticker resource. After the download of the audio resource is complete, the audio data is rendered and loaded into the video data, and at the same time as audio playback is performed, the downloaded sticker resource is dynamically rendered, allowing the sticker data to be dynamically added to the current video data, and the effect presented to the user is that the stickers have been dynamically added.

[0093] Of course, if audio data is added 2 seconds after the start of video data playback, and sticker data is added 4 seconds after the start of video data playback, and the system detects that the video data has been played up to 1.9 seconds, it determines the necessary audio resources for the next playback position (i.e., the position corresponding to 2 seconds) from the non-essential rendering resources, downloads them, performs rendering on the downloaded audio resources, and plays the video data and audio data simultaneously. Similarly, if the system detects that the video data has been played up to 3.9 seconds, it determines the necessary sticker resources for the next playback position (i.e., the position corresponding to 4 seconds) from the non-essential rendering resources, downloads them, performs rendering on the downloaded sticker resources, and plays the video data and audio data while dynamically displaying the sticker data on the video screen.

[0094] To make it clearer, if factors such as network latency are present, the playback time after the media rendering resources are first dynamically loaded may be later than the actual playback node set in the rendering template. For example, in the above example, if the sticker display time is from 4s to 8s, and it is detected that the video data is played until 3.9s, the necessary sticker resources should be determined from the non-essential rendering resources for the next playback position (i.e., the position corresponding to 4s), downloaded, and played. However, due to network factors, the sticker resource download may not be completed by 4s, and after the sticker resource download is completed at 4.2s, the sticker effect can be played further until the sticker display is canceled at 8s.

[0095] The media resource rendering method according to this embodiment detects the current playback position to determine the target resources that need to be loaded at the next playback position. This allows for the dynamic determination of necessary target resources from non-essential rendering resources, prioritizing the loading of media resources that will be played from now on. This eliminates the need to sequentially download all non-essential rendering resources, saving loading time. Simultaneously, the determined target resources are processed sequentially according to a pre-set priority, enabling incremental rendering of non-essential rendering resources. Target resources can be dynamically added to the first media data, achieving the effect of dynamic loading and enhancing the user's preview experience of media effects.

[0096] In this embodiment, a method for rendering media resources that can be used in the above-mentioned computer devices, such as mobile phones, tablet PCs, and computers, is provided. Figure 3 is a flowchart of the media resource rendering method according to an embodiment of this disclosure, and as shown in Figure 3, the flow includes the following steps.

[0097] Step S301: In the media editing interface, retrieve multiple media rendering resources in the rendering template.

[0098] Specifically, step S301 above includes the following:

[0099] Step S3011: In the media editing interface, retrieve resource description information in the rendering template.

[0100] Resource description information is a download path description for remote resource packs, predefined by the engineer in the rendering template protocol. All media rendering resources included in a rendering template have corresponding resource description information. Specifically, when a rendering template is obtained, the resource description information for all media rendering resources can be obtained by analyzing the protocol information corresponding to the rendering template.

[0101] Step S3012: Based on the resource description information, retrieve the corresponding media remote resource.

[0102] Media remote resources are media rendering resources stored on a remote server. Based on the resource description information, the download path for media remote resources can be determined, and based on that download path, the system can be positioned to the corresponding remote server, thereby allowing all media remote resources to be retrieved from the remote server pointed to by the download address.

[0103] Step S3013: Perform a conversion on the media remote resource according to a pre-configured conversion protocol to obtain a media rendering resource.

[0104] The pre-configured conversion protocol is a data conversion protocol between the remote server and the rendering template. Media remote resources may originate from different servers, and the data definition may differ depending on the remote server. In this case, to ensure data consistency, data conversion can be performed on various media remote resources according to the pre-configured conversion protocol to obtain media rendering resources that the rendering template can identify. These media rendering resources can then be entered into the rendering template to prepare them for use in subsequent rendering.

[0105] Step S302: First media data is generated based on multiple media rendering resources. Here, the first media data includes a first rendering resource set, which includes at least some media rendering resources. A detailed explanation is provided in the relevant descriptions corresponding to the above embodiments and is not explained further here.

[0106] In step S303, the first media data is played back in the media editing interface. For a detailed explanation, please refer to the relevant description corresponding to the above embodiment, and no further explanation is provided here.

[0107] In step S304, during the process of playing the first media data, the playback effects of the second rendering resource set are added in real time. Here, the second rendering resource set includes media rendering resources other than the first rendering resource set among multiple media rendering resources. A detailed explanation is provided in the relevant description corresponding to the above embodiment and is not explained further here.

[0108] The media rendering resource rendering method according to this embodiment performs protocol conversion on the acquired media remote resource to generate a media rendering resource whose rendering is supported by the current rendering template, thereby improving compatibility with various remote resources.

[0109] As one specific application example of the embodiments of this disclosure, when media effects are reproduced based on a rendering template, the implementation method is as follows:

[0110] 1) Download the essential resources in the prepared rendering template (i.e., NLEModel) early, which are related to audio-video rendering and, for example, the main track video clip that needs to be played first. Here, the essential resources in the rendering template are identified according to predefined rules.

[0111] 2) After downloading the required resources, the NLEModel is directly passed to the rendering layer (i.e., the NLEMedia layer), which performs the rendering process of the required resources. The NLEMedia layer can then render the main track video based on the existing required resources, i.e., it displays the video screen to the user.

[0112] 3) In the NLEModel template, there are many non-essential rendering resources. If all of them are downloaded sequentially at once, it may cause slow resource loading. In this case, each non-essential rendering resource, such as other video resources, audio resources, sticker / filter resources, and various style resources, should be downloaded sequentially based on a pre-configured priority. This pre-configured priority can support service configuration.

[0113] Specifically, when playing a video, non-essential rendering resources are prioritized based on the current playback position of the video, and non-essential rendering resources related to the content to be played are downloaded first. Furthermore, resources are selected based on the priority of various types of non-essential rendering resources, and if, for example, the video resource has the highest priority, that video resource is downloaded first.

[0114] 4) The downloaded non-essential rendering resources are rendered by the NLEMedia layer, and effects corresponding to the non-essential rendering resources are added to the audio-video rendering engine for playback.

[0115] This approach was used to set up multiple sets of experiments, and the experimental results shown in Figure 4 were obtained. By analyzing the experimental results, it was observed that the average time required for each rendering template was optimized by 455ms, the effect download flow was optimized by 20.2%, and the overall application flow of the rendering templates was optimized by 7.1%. As can be seen from this, this approach increased the loading speed of rendering templates and improved the user experience.

[0116] In this embodiment, a media resource rendering device is further provided, which is used to implement the above embodiment and preferred embodiments, and what has already been described will not be explained further. As used below, the term "module" can implement a combination of software and / or hardware with a predetermined function. The devices described in the following embodiments are preferably implemented in software, but implementation in hardware, or a combination of software and hardware, is also possible and conceived.

[0117] This embodiment provides a rendering device for media rendering resources, and includes the following, as shown in Figure 5.

[0118] The resource acquisition module 401 is used in the media editing interface to acquire multiple media rendering resources in the rendering template.

[0119] The generation module 402 is used to generate first media data based on multiple media rendering resources, wherein the first media data includes a first rendering resource set, and the first rendering resource set includes at least some media rendering resources.

[0120] The playback module 403 is used to play the first media data in the media editing interface.

[0121] The additional module 404 is used in the process of playing back the first media data to add the playback effects of the second rendering resource set in real time. Here, the second rendering resource set includes media rendering resources other than the first rendering resource set among multiple media rendering resources.

[0122] In some optional embodiments, the generation module 402 is An identification unit for identifying at least one essential rendering resource from multiple media rendering resources, It may also include a resource processing unit for processing essential rendering resources and generating first media data, where the first media data includes essential rendering resources.

[0123] In some optional embodiments, the identification unit is An identifier identification unit for determining whether or not the required identifiers are being carried to each media rendering resource, It may include a required determination unit for determining one or more media rendering resources carrying required identifiers as required rendering resources.

[0124] In some optional embodiments, the additional module 404 is The system may include a data generation unit for generating second media data based on first media data and a second rendering resource set, where the second media data includes the first media data and at least some media rendering resources in the second rendering resource set that are added in real time.

[0125] In some embodiments, the second rendering resource set includes non-essential rendering resources from among multiple media rendering resources, and the data generation unit is The rendering subunit may include a rendering subunit that processes one or more non-essential rendering resources within a predetermined time period after processing essential rendering resources to obtain processing results, and then generates and plays back second media data, which includes the non-essential rendering resources, based on the first media data, using the processing results as a foundation.

[0126] In some embodiments, the rendering subunit is further used to determine the playback position of the first media data when the first media data is played for a predetermined duration, to determine the target resources required for the next playback position from non-essential rendering resources based on the playback position, to process the target resources according to a predetermined priority, to generate second media data corresponding to the target resources, and to play the second media data based on the first media data.

[0127] In some optional embodiments, the rendering subunit is further used to obtain the load time corresponding to each non-essential rendering resource, determine the target playback position of each non-essential rendering resource based on the first media data based on the load time, compare the target playback position with the next playback position, and screen the non-essential rendering resources for the target resources required for the next playback position based on the comparison result.

[0128] In some optional embodiments, the rendering subunit is further used to construct a plurality of download tasks corresponding to a target resource, to prioritize the plurality of download tasks based on a predetermined priority, to obtain a priority prioritization result, to trigger each download task sequentially according to the priority prioritization result, and to generate second media data.

[0129] In some optional embodiments, the rendering subunit is further used to detect whether the number of download tasks exceeds a preset concurrent execution amount, and if the number of download tasks exceeds the preset concurrent execution amount, to determine one or more current download tasks according to the preset concurrent execution amount, to trigger one or more current download tasks sequentially according to the priority ordering result, and to obtain second media data corresponding to each download task.

[0130] In some optional embodiments, the resource acquisition module 401 is: An information acquisition unit for obtaining resource description information in the rendering template within the media editing interface, A remote resource acquisition unit for obtaining the corresponding media remote resource based on the resource description information, It may also include a resource conversion unit that performs conversions on media remote resources according to a pre-configured conversion protocol to obtain media rendering resources.

[0131] Further functional descriptions of each module and unit are the same as those in the corresponding embodiments described above and will not be explained further here.

[0132] In this embodiment, the media resource rendering apparatus is represented in the form of a functional unit, wherein the unit refers to an ASIC circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0133] The media resource rendering device according to this embodiment generates and plays first media data using a portion of multiple media rendering resources acquired in the media editing interface. After playing the first media data, it processes other media rendering resources and adds playback effects corresponding to those of the other media rendering resources to the first media data in real time. As a result, media resources in the rendering template do not need to wait until all downloads are complete during rendering; they can be downloaded and rendered as needed, thus reducing the loading time of media resources. Furthermore, during the rendering process, some media rendering resources can be processed preferentially to show the corresponding media data to the user, reducing the user's waiting time. Other media rendering resources can be played while rendering, thereby dynamically adding corresponding playback effects after playing the first media data, reducing the impact on the presentation of media effects, and enhancing the user's preview experience of media effects.

[0134] The embodiments of this disclosure further provide a computer device having a media resource rendering device as shown in Figure 5 above.

[0135] Referring to Figure 6, which is a schematic diagram of the structure of a computer device according to an optional embodiment of the present disclosure, as shown in Figure 6, the computer device includes one or more processors 10, memory 20, and interfaces for connecting each component, including a high-speed interface and a low-speed interface. Each component communicates with one another using different buses and may be mounted on a common mainboard or in other ways as needed. The processors can process instructions executed within the computer device, including instructions in or stored in memory and used to display GUI graphics information on an external input / output device (e.g., a display device coupled to an interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses may be used with multiple memories and multiple memory. Similarly, multiple computer devices may be connected, each providing a portion of the necessary operations (e.g., a server array, a set of blade servers, or a multiprocessor system). In Figure 6, one processor 10 is used as an example.

[0136] The processor 10 may be a central processing unit, a network processor, or a combination thereof. Here, the processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a composite programmable logic device, a field-programmable logic gate array, a general-purpose array logic, or any combination thereof.

[0137] Here, the memory 20 stores instructions that can be executed by at least one processor 10, and causes at least one processor 10 to execute the method shown in the above embodiment.

[0138] Memory 20 may include a program storage area and a data storage area, where the program storage area can store an operating system and application programs necessary for at least one function, and the data storage area can store data created based on the use of the computer equipment. Memory 20 may also include high-speed random-access memory and may further include non-temporary memory, such as at least one disk memory device, flash memory device, or other non-temporary solid-state memory device. In some optional embodiments, memory 20 optionally includes memory installed remotely from the processor 10, and these remote memories can be connected to the computer equipment via a network. Examples of the network include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0139] The memory 20 may include volatile memory, such as random access memory, and the memory may also include non-volatile memory, such as flash memory, a hard disk, or a solid-state hard disk. The memory 20 may further include combinations of the above types of memory.

[0140] The computer equipment further includes an input device 30 and an output device 40. The processor 10, memory 20, input device 30, and output device 40 can be connected by a bus or other means, with Figure 6 showing a bus connection as an example.

[0141] The input device 30 can receive input numerical or character information and generate key signal inputs related to user settings and function control of the computer equipment, such as a touchscreen, keypad, mouse, trackpad, touchpad, indicator lever, one or more mouse buttons, trackball, joystick, etc. The output device 40 may include a display device, auxiliary lighting device (e.g., LEDs), and haptic feedback device (e.g., vibration motor), etc. The display device includes, but is not limited to, liquid crystal displays, light-emitting diodes, displays, and plasma displays. In some optional embodiments, the display device may be a touchscreen.

[0142] The computer equipment further includes a communication interface for data communication between the computer equipment and other equipment or a communication network.

[0143] Embodiments of the present disclosure further provide computer-readable storage media, and the methods according to embodiments of the present disclosure may be implemented in hardware, firmware, or as recordable on a storage medium, or as computer code stored on a local storage medium and downloaded over a network to an original remote storage medium or non-temporary machine-readable storage medium, thereby enabling the methods described herein to be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Here, the storage medium may be a magnetic disk, an optical disk, read-only memory, random access memory, flash memory, a hard disk, or a solid-state hard disk, and furthermore, the storage medium may include combinations of the above types of memory. To understand that a computer, processor, microprocessor controller, or programmable hardware includes a storage component capable of storing or receiving software or computer code, and when the software or computer code is accessed and executed by the computer, processor, or hardware, the methods described in the embodiments are implemented.

[0144] Embodiments of the present disclosure further provide a computer program which includes computer instructions, and when the computer instructions are executed by a processor (e.g., processor 10), the method described in the above embodiment is realized.

[0145] While embodiments of this disclosure have been described with reference to the drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of this disclosure, all of which are limited by the appended claims.

Claims

1. A method for rendering media resources, In the media editing interface, retrieving multiple media rendering resources in the rendering template, A first media data is generated based on the plurality of media rendering resources, and the first media data includes a first rendering resource set which includes at least some of the media rendering resources. Playing the first media data in the media editing interface, A method for rendering media resources, comprising, in the process of playing back the first media data, adding in real time the playback effect of a second rendering resource set, which includes other media rendering resources among the plurality of media rendering resources other than the first rendering resource set.

2. The first rendering resource set includes essential rendering resources from among the plurality of media rendering resources, Generating the first media data based on the aforementioned multiple media rendering resources is: Identifying at least one essential rendering resource from the aforementioned multiple media rendering resources, The method according to claim 1, comprising processing the at least one required rendering resource to generate the first media data including the at least one required rendering resource.

3. Identifying the at least one required rendering resource from the aforementioned multiple media rendering resources is: Identifying whether or not the essential identifiers are being carried to each of the aforementioned multiple media rendering resources, The method according to claim 2, further comprising determining one or more media rendering resources carrying the required identifier as the required rendering resources.

4. In the process of playing back the first media data, adding the playback effects of the second rendering resource set in real time is: The method according to any one of claims 1 to 3, comprising generating second media data based on the first media data and the second rendering resource set, wherein the second media data comprises the first media data and at least some media rendering resources in the second rendering resource set that are added in real time.

5. The second rendering resource set includes non-essential rendering resources from among the multiple media rendering resources, Generating the second media data based on the first media data and the second rendering resource set is: The method according to claim 4, comprising processing one or more non-essential rendering resources within a predetermined time period after processing essential rendering resources to obtain processing results, and generating and playing back second media data including the non-essential rendering resources based on the processing results and the first media data.

6. After processing the essential rendering resources, processing one or more of the non-essential rendering resources within the predetermined time period to obtain the processing results, and generating and playing back the second media data including the non-essential rendering resources based on the processing results and the first media data, is: When the first media data is played back for the predetermined time length, the playback position of the first media data is determined, Based on the aforementioned playback position, the target resources required for the next playback position are determined from the aforementioned non-essential rendering resources, The process involves performing processing on the target resource according to a pre-set priority and generating the second media data corresponding to the target resource. The method according to claim 5, further comprising playing back the second media data based on the first media data.

7. Based on the aforementioned playback position, determining the target resources required for the next playback position from the non-essential rendering resources is: Obtain the load time corresponding to each of the aforementioned non-essential rendering resources, and determine the target playback position of each of the aforementioned non-essential rendering resources based on the first media data, The method according to claim 6, comprising comparing the target playback position with the next playback position, and screening the target resources required for the next playback position from the non-essential rendering resources based on the comparison result.

8. Performing processing on the target resource according to the pre-set priority and generating the second media data corresponding to the target resource is: To construct multiple download tasks corresponding to the aforementioned target resource, Based on the aforementioned pre-set priorities, the multiple download tasks are prioritized and the priority prioritization results are obtained. The method according to claim 6, further comprising triggering the plurality of download tasks in order according to the priority ordering result and generating the second media data.

9. Triggering each of the download tasks in order according to the priority ordering result and generating the second media data is: The system detects whether the number of the aforementioned download tasks exceeds a predetermined limit for simultaneous execution, In response to the determination that the number of download tasks exceeds the predetermined concurrent execution amount, one or more current download tasks are determined according to the predetermined concurrent execution amount. The method according to claim 8, further comprising triggering one or more of the current download tasks in order according to the priority ordering result, and obtaining the second media data corresponding to each download task.

10. In the media editing interface, obtaining the multiple media rendering resources in the rendering template means that In the media editing interface, the resource description information in the rendering template is obtained, Based on the aforementioned resource description information, the corresponding media remote resource is obtained, The method according to any one of claims 1 to 3, comprising performing a conversion on the media remote resource according to a pre-configured conversion protocol to obtain the media rendering resource.

11. A rendering device for media resources, A resource acquisition module for obtaining multiple media rendering resources in a rendering template within the media editing interface, A generation module for generating first media data based on the plurality of media rendering resources, wherein the first media data includes a first rendering resource set that includes at least some of the media rendering resources, The media editing interface includes a playback module for playing the first media data, A media resource rendering apparatus, comprising an additional module for adding in real time the playback effect of a second rendering resource set, which includes other media rendering resources from among the plurality of media rendering resources other than the first rendering resource set, in the process of playing back the first media data.

12. Computer equipment, A computer device comprising memory and a processor, wherein the memory and the processor are communicated with each other, computer instructions are stored in the memory, and the processor performs the method for rendering a media resource according to any one of claims 1 to 10 by executing the computer instructions.

13. A computer-readable storage medium, wherein computer instructions are stored in the computer-readable storage medium, and when the computer instructions are executed by a computer, the computer causes the computer to execute the media resource rendering method described in any one of claims 1 to 10.

14. A computer program that includes a computer instruction, and when the computer instruction is executed by the computer, causes the computer to execute the media resource rendering method described in any one of claims 1 to 10.

Images