Method and apparatus for processing media content, device and storage medium

Abstract

Provided in the embodiments of the present disclosure are a method and apparatus for processing media content, a device, and a computer-readable storage medium. The method comprises: receiving a frame size change instruction associated with first media content, the first media content comprising a first image and a first group of media materials; acquiring second media content, the second media content comprising a second image and a second group of media materials, the second image being determined from the first image on the basis of frame size information indicated by the frame size change instruction, and the second group of media materials being determined on the basis of the type of the first group of media materials; and triggering displaying the second media content.

Description

Methods, apparatus, devices and storage media for processing media content

[0001] This application claims priority to Chinese Patent Application No. 202411848228.6, filed on December 13, 2024, entitled "Method, Apparatus, Device and Storage Medium for Processing Media Content", the entire contents of which are incorporated herein by reference. Technical Field

[0002] The exemplary embodiments disclosed herein relate generally to the field of computers, and more particularly to methods, apparatus, devices, and computer-readable storage media for processing media content. Background Technology

[0003] With the development of computer technology, the internet has become an important platform for people to obtain various kinds of information. Media content is an important form of information carrier. Some platforms, for example, allow users to add media materials (such as stickers) to image or video files to create richer media content. In some scenarios, the aspect ratio of the media content may not match the display components, which necessitates cropping the media content. Summary of the Invention

[0004] In a first aspect of this disclosure, a method for processing media content is provided. The method includes: receiving a frame change instruction associated with first media content, the first media content including a first image and a first set of media materials; acquiring second media content, the second media content including a second image and a second set of media materials, the second image being determined from the first image based on frame information indicated by the frame change instruction, and the second set of media materials being determined based on the type of the first set of media materials; and triggering the display of the second media content.

[0005] In a second aspect of this disclosure, an apparatus for processing media content is provided. The apparatus includes: a receiving module configured to receive a frame change instruction associated with first media content, the first media content including a first image and a first set of media materials; an acquiring module configured to acquire second media content, the second media content including a second image and a second set of media materials, the second image being determined from the first image based on frame information indicated by the frame change instruction, and the second set of media materials being determined based on the type of the first set of media materials; and a triggering module configured to trigger the display of the second media content.

[0006] In a third aspect of this disclosure, an electronic device is provided. The device includes at least one processor; and at least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor. When executed by the at least one processor, the instructions cause the device to perform the method of the first aspect.

[0007] In a fourth aspect of this disclosure, a computer-readable storage medium is provided. The computer-readable storage medium stores computer-executable instructions that can be executed by a processor to implement the method of the first aspect.

[0008] In a fifth aspect of this disclosure, a computer program product is provided, which is tangibly stored in a computer storage medium and includes computer-executable instructions that, when executed by a device, cause the device to perform the method of the first aspect.

[0009] It should be understood that the content described in this content section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0010] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0011] Figure 1 shows a schematic diagram of an example environment in which embodiments of the present disclosure may be implemented;

[0012] Figures 2A to 2F illustrate example interfaces according to some embodiments of the present disclosure;

[0013] Figure 3 shows a flowchart of an example process for generating a music video according to some embodiments of the present disclosure;

[0014] Figure 4 shows a schematic structural block diagram of an example apparatus for generating music videos according to some embodiments of the present disclosure; and

[0015] Figure 5 shows a block diagram of an electronic device capable of implementing several embodiments of the present disclosure. Detailed Implementation

[0016] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0017] It should be noted that the headings of any section / subsection provided herein are not limiting. Various embodiments are described throughout this document, and embodiments of any type may be included under any section / subsection. Furthermore, embodiments described in any section / subsection may be combined in any way with any other embodiments described in the same section / subsection and / or different sections / subsections.

[0018] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below. The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0019] The embodiments of this disclosure may involve user data, data acquisition, and / or use. All of these aspects comply with applicable laws, regulations, and relevant provisions. In the embodiments of this disclosure, all data collection, acquisition, processing, manipulation, forwarding, and use are conducted with the user's knowledge and confirmation. Accordingly, in implementing the embodiments of this disclosure, the type, scope of use, and usage scenarios of any data or information that may be involved should be communicated to the user and their authorization obtained in accordance with relevant laws and regulations through appropriate means. The specific methods of notification and / or authorization may vary depending on the actual situation and application scenario, and the scope of this disclosure is not limited in this respect.

[0020] In this specification and the embodiments, any processing of personal information will be carried out only under the premise of legality (such as obtaining the consent of the personal information subject, or being necessary for the performance of a contract), and will only be carried out within the scope stipulated or agreed upon. A user's refusal to process personal information other than that necessary for basic functions will not affect the user's use of basic functions.

[0021] As mentioned above, the aspect ratio of media content may not match that of the display components, necessitating cropping of the media content. Traditional cropping methods typically involve directly cropping the corresponding area based on the component's aspect ratio, which can lead to the loss of crucial information in the media content, significantly impacting the viewing experience.

[0022] The embodiments of this disclosure propose a generation scheme. The scheme includes: receiving a frame change instruction associated with first media content, the first media content including a first image and a first set of media materials; acquiring second media content, the second media content including a second image and a second set of media materials, the second image being determined from the first image based on frame information indicated by the frame change instruction, and the second set of media materials being determined based on the type of the first set of media materials; and triggering the display of the second media content.

[0023] On the one hand, embodiments of this disclosure can crop images in media content based on aspect ratio information; on the other hand, embodiments of this disclosure can also determine the retention strategy for materials based on the type of materials in the original media content. In this way, embodiments of this disclosure can improve the quality of media content after aspect ratio changes.

[0024] The following section provides a detailed description of various example implementations of this scheme, with reference to the accompanying drawings.

[0025] Example Environment

[0026] Figure 1 illustrates a schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented. As shown in Figure 1, the example environment 100 may include an electronic device 110.

[0027] In this example environment 100, electronic device 110 may run application 120 that supports the generation of music videos. Application 120 may be any suitable type of application, examples of which may include, but are not limited to, video applications, live streaming applications, or other suitable applications capable of providing music video generation services. User 140 may interact with application 120 via electronic device 110 and / or its attached devices.

[0028] In environment 100 of Figure 1, if application 120 is active, electronic device 110 can use application 120 to present interface 150 for supporting the generation of music videos.

[0029] In some embodiments, electronic device 110 communicates with server 130 to provide services to application 120. Electronic device 110 can be any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, handheld computers, portable gaming terminals, VR / AR devices, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio receivers, e-book devices, gaming devices, or any combination thereof, including accessories and peripherals of these devices or any combination thereof. In some embodiments, electronic device 110 can also support any type of user-facing interface (such as "wearable" circuitry).

[0030] Server 130 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms. Server 130 may include, for example, computing systems / servers such as mainframes, edge computing nodes, computing devices in a cloud environment, etc. Server 130 can provide background services for application 120 in electronic device 110 that supports music and video generation.

[0031] A communication connection can be established between server 130 and electronic device 110. This communication connection can be established via wired or wireless means. The communication connection may include, but is not limited to, Bluetooth, mobile network, Universal Serial Bus (USB), and Wireless Fidelity (WiFi) connections; the embodiments of this disclosure are not limited in this respect. In the embodiments of this disclosure, server 130 and electronic device 110 can achieve signaling interaction through the communication connection between them.

[0032] It should be understood that the structure and function of the various elements in environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of this disclosure.

[0033] The following description will continue with reference to the accompanying drawings, which will provide some exemplary embodiments of this disclosure.

[0034] Example Interaction

[0035] Figure 2A illustrates a first media content 205 according to some embodiments of the present disclosure. As shown in Figure 2A, the first media content 205 may include a first image 210 and a set of media materials, such as media material 220 and media material 225. As an example, the first media content 205 may correspond to a first aspect ratio, such as 4:3.

[0036] In some embodiments, the first image 210 may include, for example, a photograph, a video, or an image captured in real time from a viewfinder.

[0037] Furthermore, media assets 220 and 225 can be overlaid on the first image 210, for example. As an example, such media assets 220 and 225 can be added based on user editing operations. For instance, media assets 220 and 225 may include assets generated based on effects applied by the user.

[0038] In some embodiments, media assets may also include stickers used to present information within the media content. For example, such stickers may include, but are not limited to: time stickers, weather stickers, magnifying glass stickers, text stickers, lyric stickers, etc.

[0039] Additionally, media materials may also include widgets. As an example, widgets may have corresponding functional attributes, such as poll widgets, video reply widgets, points of interest widgets, live stream preview widgets, and video sharing widgets. In some scenarios, widgets can also be understood as a special type of sticker.

[0040] In some scenarios, such stickers and widgets are interactive. For example, stickers or widgets can be configured with corresponding interaction logic. For instance, users can click on a sticker or widget in the first media content 205 to trigger access to other interfaces or perform pre-defined interactive actions.

[0041] Taking the voting widget as an example, users can perform the corresponding voting action by clicking the voting widget in the first media content 205.

[0042] In some embodiments, the media material may also include other suitable types of interface components. Such interface components may be configured to form part of the first media content 205 together with the first image 210.

[0043] As mentioned above, changes in the aspect ratio of media content are a common scenario. For example, when the display component used to present media content does not match the current aspect ratio of the media content, the aspect ratio of the media content needs to be adjusted.

[0044] Taking Figure 2B as an example, interface 200B may include a widget 230. As an example, such a widget 230 can be associated with application 120 and also supports displaying information associated with application 120. In some scenarios, widget 230 can be displayed on the system desktop, typically having a fixed aspect ratio. As an example, the aspect ratio corresponding to widget 230 is 1:1.

[0045] In some scenarios, when widget 230 is configured to display first media content 205, the aspect ratio of the first media content 205 needs to be adjusted. For example, the first media content 205 may be a widget associated with a user in application 120, which may display works published by that user. As an example, when the user publishes the first media content 205, widget 230 needs to display second media content 235 obtained by adjusting the aspect ratio of the first media content 205.

[0046] As described above, traditional image aspect ratio adjustments are usually based on center-point cropping, which can lead to fragmented display of image content and media materials, severely impacting information retrieval efficiency. For example, in the case of the first image 210 including a person's face, center cropping may result in the user's face not being fully displayed.

[0047] As shown in Figure 2B, embodiments of this disclosure can take into account the content of the first image 210 and the type of media material during the aspect ratio adjustment process, and can accordingly obtain a second media content 235 that matches the target aspect ratio (e.g., 1:1).

[0048] On one hand, the second media content 235 may include a second image obtained by cropping the first image 210. As an example, the second image may correspond to a aspect ratio supported by the widget 230 (e.g., 1:1).

[0049] On the other hand, the second media content may also retain at least some of the media materials (also referred to as the second set of media materials) in the first media content 205. As an example, media material 220 and media material 225 (e.g., a hat sticker) may be retained.

[0050] As will be detailed below, the second set of media materials that are retained can be determined based on the type of media materials in the first media content 205. As an example, if the first media content 205 includes interactive widgets, the second media content 230 can retain those interactive widgets regardless of whether they are retained within the cropping range.

[0051] By taking into account the type of media material during the frame adjustment process, embodiments of this disclosure can improve the quality of media content after frame change, thereby enhancing the completeness of information display.

[0052] In addition to the widget scenario shown in Figure 2B, the embodiments of this disclosure are also applicable to other suitable aspect ratio adjustment scenarios.

[0053] For example, during media content editing, a user can select the first media content 205 in the first aspect ratio. Furthermore, the user can initiate a canvas adjustment request for the first media content 205, such as adjusting its aspect ratio from 4:3 to 1:1.

[0054] As another example, in the scenario of publishing media content, users can select the first media content 205 with the first aspect ratio. Furthermore, depending on the type of work being published or the platform, the aspect ratio of the first media content 205 can be automatically adjusted, or multiple preset aspect ratios can be provided to support user selection. For example, if certain types of works only support a 1:1 aspect ratio, then when a user uses the first media content 205 to publish that type of work, the aspect ratio of the first media content 205 can be automatically adjusted.

[0055] As another example, in scenarios where a camera is used for framing, the user can obtain images captured by the camera in real time. In some scenarios, the user can also adjust the camera's field of view, thereby causing a corresponding change in the image's aspect ratio.

[0056] In other examples, certain media display components have pre-defined aspect ratios. For instance, in a news feed browsing interface, the aspect ratio of the media content's display component can be fixed in advance. When the media content to be played does not match the aspect ratio of this display component, the aspect ratio of the media content also needs to be adjusted.

[0057] The following will describe the specific process of aspect ratio adjustment in detail with reference to Figures 2C to 2F. It should be understood that such aspect ratio adjustment can be performed on appropriate electronic devices, such as server 130 or electronic device 110 that displays the media content after the canvas change.

[0058] In some embodiments, electronic device 110 and / or server 130 may receive a frame change instruction associated with the first media content 205. Such a frame change instruction may be generated automatically by the system or may be generated based on user interaction.

[0059] Furthermore, the electronic device 110 and / or the server 130 can determine the cropping range in the first image 210 of the first media content 205 based on the frame information indicated by the frame change instruction.

[0060] In some embodiments, the electronic device 110 and / or the server 130 may determine the feature information of the first media content 205 and may accordingly determine a plurality of reference positions in the first media content 205.

[0061] Figure 2C illustrates several example reference locations according to some embodiments of the present disclosure. As shown in Figure 2C, electronic device 110 and / or server 130 can acquire feature information associated with first media content 205. In some embodiments, the feature information may be related to the screen content of first image 210. For example, the feature information may indicate a preset object in first image 210.

[0062] Alternatively or additionally, the feature information may be associated with a first set of media materials in the first media content. For example, the feature information may indicate the image location corresponding to the first set of media materials.

[0063] Alternatively or additionally, the feature information may also be related to the interactive information of the first media content 205. For example, special effects information may indicate the historical visual focus position of the first media content 205.

[0064] Specifically, electronic device 110 and / or server 130 can identify one or more feature points in the first media content 205 based on the feature information of the first media content 205. As shown in FIG2C, electronic device 110 and / or server 130 can identify multiple feature points, namely, feature points 204-1 to feature points 204-5 (referred to individually or collectively as feature points 204).

[0065] In some embodiments, depending on how feature points 204 are determined, such feature points may be associated with different feature point types. Furthermore, different types of feature point types may correspond to different weights.

[0066] As an example, the feature point type may include a first feature point type, which may be associated with an object of a predetermined type in the first image. As an example, the electronic device 110 and / or the server 130 may identify whether the first image includes an object of a predetermined type (e.g., a facial object), and may, for example, determine the location corresponding to the facial object as a first type of feature point.

[0067] In some examples, the weights corresponding to the first type of feature points can also be associated with the type of object. For example, a facial object may have the highest weight. Furthermore, objects such as food, clothing, and buildings may have lower weights than facial objects.

[0068] Taking Figure 2C as an example, feature point 204-2 may correspond to a first type of feature point, which may, for example, correspond to the center point of a facial object in the first image.

[0069] In some embodiments, the feature point type may include a second feature point type associated with media assets in the first media content 205. As an example, the electronic device 110 and / or server 130 may determine the positions corresponding to each media asset in the first media content 205 as second-type feature points. For example, feature points 240-1, 240-3, and 240-4 may correspond to the second-type feature points, which may correspond to the positions of media assets such as stickers or pendants in the image.

[0070] In some examples, the weight of the second type of feature points may be lower than the weight of the first type of feature points.

[0071] In some embodiments, the feature point type may include a third feature point type, which may be associated with the interaction information of the first media content 205. As an example, the electronic device 110 and / or the server 130 may determine the historical visual focus position of the first media content 205 as a third type of feature point. Alternatively, the electronic device 110 and / or the server 130 may also use the center position of the first media content 205 as a third type of feature point.

[0072] Furthermore, the electronic device 110 and / or the server 130 can also determine the visual dwell time of the first media content 205, and can determine the weight of the third type of feature points based on the visual dwell time. For example, the longer the visual dwell time, the higher the weight of the third type of feature points can be.

[0073] Alternatively or additionally, the feature point type may also include a fourth feature point type, which may be associated with the content of the first image. For example, electronic device 110 and / or server 130 may identify regions with strong visual contrast and / or high information density in the first image, and may identify fourth type feature points corresponding to these regions.

[0074] As an example, electronic device 110 and / or server 130 can identify image regions in the first image with strong contrast and high saturation, and can determine the center point of such regions as fourth-type feature points. As yet another example, electronic device 110 and / or server 130 can also identify regions in the image with a large number of entity objects. Taking Figure 2C as an example, electronic device 110 and / or server 130 can determine that the bottom region of the first image has a high density of entity objects (e.g., clouds), and can determine the center point of such region as a fourth-type feature point, namely, feature point 240-5.

[0075] In some embodiments, the number of feature points of each type may be set to a predetermined upper limit. When the number of a certain type of feature exceeds the upper limit, the electronic device 110 and / or the server may, for example, randomly sample feature points of that type.

[0076] Furthermore, the electronic device 110 and / or the server 130 can determine the target location based on multiple feature points 240 and their corresponding weight information. As an example, the electronic device 110 and / or the server 130 can determine the weighted center position of multiple feature points 240 as the target location.

[0077] As shown in Figure 2D, the electronic device 110 and / or server 130 can determine the target position 245 based on, for example, multiple feature points 240. In some embodiments, the target position 245 can be determined as the center point of the image area to be cropped. In this way, embodiments of this disclosure can ensure that important image content of the first media content can be preserved in the cropping area.

[0078] Furthermore, the electronic device 110 and / or the server 130 can determine the cropping area 250 based on the target location 245 and the aspect ratio to be cropped (e.g., 1:1), and can then determine a second image that matches the aspect ratio from the first image 210.

[0079] Specifically, the electronic device 110 and / or the server 130 may, for example, use the target position 245 as the center point and scale the cropping area based on the image to be cropped (e.g., 1:1) until the cropping area is the same length or width as the first image.

[0080] Therefore, electronic device 110 and / or server 130 can perform adaptive image cropping based on the content of the first image, thereby preserving the image area with higher weight.

[0081] In some embodiments, during the process of determining the cropping region, the electronic device 110 and / or the server 130 may also check whether the cropping region completely preserves the image region corresponding to the preset object in the first image. For example, if the cropping region only preserves a portion of the facial object in the first image, the electronic device 110 and / or the server 130 may discard the cropping region and may redetermine the target location.

[0082] As an example, during the process of redetermining the target location, electronic device 110 and / or server 130 may resample various types of feature points to redetermine the center point of the cropping region. As another example, during the process of redetermining the target location, electronic device 110 and / or server 130 may, for example, disregard feature points with reduced weights, or further reduce the weights of these feature points, to prioritize the preservation of feature points with higher weights.

[0083] After the cropping area is determined, the electronic device 110 and / or the server 130 can accordingly acquire the second image to be retained. The dynamic adaptation of media material during aspect ratio changes will be further described below.

[0084] In some embodiments, the electronic device 110 and / or the server 130 may implement different adaptation strategies for different types of media materials.

[0085] As an example, media materials can be divided into information display materials and interactive materials. Further, information display materials can be further divided into media materials associated with the visual content and media materials independent of the visual content.

[0086] Using Figure 2E as an example, media material 225 may include sticker material generated based on special effects, such as a hat. This sticker material may belong to the information display category. Furthermore, this sticker material is associated with the content of the image; that is, the sticker material is attached to the head position in the image.

[0087] In this case, if the target image content (e.g., head) corresponding to media material 225 is retained in the second image, then media material 225 will be retained in the second media content, regardless of whether the cropping area 250 completely covers media material 225.

[0088] In addition, the electronic device 110 and / or the server 130 can also adjust the display position of the media material 225 based on the transformation relationship between the first image and the second image, so that the media material 225 is displayed at the target screen content (e.g., the head) in the second image.

[0089] In some embodiments, the electronic device 110 and / or the server 130 may determine a transformation matrix between the first image and the second image, and may determine the display position of the media material 225 in the second image based on the transformation matrix.

[0090] As an example, the transformation matrix can be represented as:

[0091] Where Scale X represents the scaling ratio in the X direction, Scale Y represents the scaling ratio in the Y direction, Skew X represents the skew in the X direction, Skew Y represents the skew in the Y direction, Translate X represents the translation of the center point in the X direction, and Translate Y represents the translation of the center point in the Y direction.

[0092] Conversely, if the target screen content corresponding to the information display material is not retained in the second image, then the information display material may also not be retained. For example, if the original media content also includes sticker material corresponding to the hand position, and the hand position is not retained in the second image, then the sticker material corresponding to the hand position may also not be retained.

[0093] In some embodiments, the electronic device 110 and / or the server 130 may also determine a second piece of content, independent of the first image, from the first set of media materials. Further, the electronic device 110 and / or the server 130 may determine whether the second piece of content meets preset conditions.

[0094] Furthermore, if the second material is information display material unrelated to the image content, the electronic device 110 and / or server 130 can determine whether it needs to be retained based on the cropping area 250. For example, if media material 220 is unrelated to the image content in the first image and is retained in the cropping area 250, it can continue to be retained in the second media content.

[0095] Furthermore, if the second material is interactive (e.g., interactive stickers or widgets), the electronic device 110 and / or server 130 may retain it in the second media content.

[0096] In some embodiments, for interactive materials, the electronic device 110 and / or server 130 may retain the relative position of the interactive materials within the media content. Specifically, the electronic device 110 and / or server 130 may determine the relative position of the interactive materials with respect to the first media content 205, and determine the display position of the interactive materials within the second media content.

[0097] Using Figure 2F as an example, the first media content may include media material 255. For example, the media material may include a challenge widget 255, which users can click to participate in the corresponding challenge interaction.

[0098] Challenge widget 255 can always be retained in the secondary media content, regardless of whether its corresponding position is within the cropped area. Using Figure 2F as an example, challenge widget 255 can be partially cropped. In this case, challenge widget 255 can still be added to the cropped secondary media content.

[0099] In some embodiments, during the adjustment of the image frame, the electronic device 110 and / or server 130 can also maintain the relative position of the interactive materials. For example, the electronic device 110 and / or server 130 can determine the distance between the challenge widget 255 and the boundary of the first media content to determine its relative position. As an example, the electronic device 110 and / or server 130 can determine the relative position of the voting widget as (1 / 3*X, 1 / 4*Y). This relative distance can, for example, mean that its distance to the left boundary is one-third of the width of the first media content, and its distance to the top boundary is one-quarter of the height of the first media content.

[0100] Accordingly, the electronic device 110 and / or the server 130 can retain their relative positions and can display them in the second media content at a relative position (1 / 3*X, 1 / 4*Y), thereby preserving the consistency of the relative position of the challenge pendant 255.

[0101] In some embodiments, the electronic device 110 and / or server 130 may also adaptively adjust the size of the media material. Specifically, the electronic device 110 and / or server 130 may determine the display size of the media material 225 based on the scaling ratio of the first image and the second image.

[0102] As an example, when the aspect ratio of the first media content changes from 4:3 to 1:1, its size can be considered to have been scaled down by 3 / 4. Correspondingly, the size of the media material retained for display in the second media content can also be adjusted to, for example, 3 / 4 of the size in the first media content, thereby improving the fit of the image.

[0103] In some embodiments, for media material associated with screen content, the electronic device 110 can also dynamically determine the scaling ratio of the media material based on the scaling ratio of the corresponding screen content. As an example, media material 225 can be associated with the header area in the first media content. If the aspect ratio adjustment process does not change the size of the header area, the size of the media material may, for example, not be scaled.

[0104] In some embodiments, for media material associated with the screen content, the electronic device 110 can also dynamically determine its scaling ratio so that it can be fully displayed in the second media content. Alternatively or additionally, for media material that falls completely within the cropping area, the electronic device 110 may, for example, not scale it, but adaptively scale media material that does not fall completely within the cropping area and needs to be retained.

[0105] In this way, embodiments of the present disclosure can achieve dynamic adaptation of media materials (such as stickers, pendants) during the frame adjustment process, thereby improving the quality of media content after frame adjustment and avoiding situations such as stickers or pendants being displayed in a fragmented manner or important pendants not being displayed.

[0106] Example process

[0107] Figure 3 illustrates a flowchart of an example process 300 for processing media content according to some embodiments of the present disclosure. Process 300 can be implemented at an electronic device 110. Process 300 will now be described with reference to Figure 1.

[0108] As shown in Figure 3, in box 310, electronic device 110 receives a frame change instruction associated with first media content, which includes a first image and a first set of media materials.

[0109] In frame 320, electronic device 110 acquires second media content, which includes a second image and a second set of media materials. The second image is determined from the first image based on the frame information indicated by the frame change instruction, and the second set of media materials is determined based on the type of the first set of media materials.

[0110] In frame 330, electronic device 110 triggers the display of second media content.

[0111] In some embodiments, process 300 further includes: determining a plurality of reference positions in the first media content based on feature information of the first media content; determining a target position based on the plurality of reference positions; and determining a second image from the first image that matches the frame information based on the target position, wherein the target position corresponds to the center point of the second image.

[0112] In some embodiments, the reference locations include multiple feature points in the first media content, and determining the target location based on the multiple reference locations includes: determining weight information corresponding to the multiple feature points based on the feature point types corresponding to the multiple feature points; and determining the target location based on the multiple feature points and the weight information.

[0113] In some embodiments, the feature point type includes at least one of the following: a first feature point type associated with an object of a preset type in the first image; a second feature point type associated with media material in the first media content; a third feature point type associated with interactive information of the first media content; and a fourth feature point type associated with the screen content of the first image.

[0114] In some embodiments, the second image retains an image region corresponding to a preset object in the first image.

[0115] In some embodiments, process 300 further includes: determining a first material of a first type from a first group of media materials, the first type indicating that the first material is associated with target screen content in a first image; and adding the first material to a second group of media materials in response to the target screen content being retained in a second image.

[0116] In some embodiments, process 300 further includes: adjusting the display position of the first material based on the transformation relationship between the first image and the second image, so that the first material is displayed at the target screen content in the second image.

[0117] In some embodiments, process 300 further includes: the first type of media material includes special effects stickers or pendants applied to preset objects in the image.

[0118] In some embodiments, process 300 further includes: determining a second material of a second type from a first group of media materials, the second type indicating that the second material has picture content independent of the first image; and adding the second material to the second group of media materials in response to the second material meeting preset conditions.

[0119] In some embodiments, process 300 further includes: determining the display position of the second material in the second media content based on the relative position of the second material with respect to the first media content.

[0120] In some embodiments, preset conditions indicate that the second material is a sticker or pendant that supports interactive operations.

[0121] In some embodiments, the display size of at least one of the media materials in the second group is determined based on the scaling ratio of the first image and the second image.

[0122] In some embodiments, triggering the display of the second media content includes: triggering the display of the second media content in a target media component corresponding to the frame information.

[0123] In some embodiments, the target media component includes at least one of the following: a widget that matches the frame information, a camera component, or a media display component.

[0124] Example devices and equipment

[0125] Embodiments of this disclosure also provide corresponding apparatus for implementing the methods or processes described above. FIG4 shows a schematic structural block diagram of an example apparatus 400 for processing media content according to certain embodiments of this disclosure. Apparatus 400 may be implemented as or included in electronic device 110. The various modules / components in apparatus 400 may be implemented by hardware, software, firmware, or any combination thereof.

[0126] As shown in Figure 4, the device 400 includes: a receiving module 410 configured to receive a frame change instruction associated with first media content, the first media content including a first image and a first set of media materials; an acquiring module 420 configured to acquire second media content, the second media content including a second image and a second set of media materials, the second image being determined from the first image based on the frame information indicated by the frame change instruction, and the second set of media materials being determined based on the type of the first set of media materials; and a triggering module 430 configured to trigger the display of the second media content.

[0127] In some embodiments, the apparatus 400 further includes a determining module configured to: determine a plurality of reference positions in the first media content based on feature information of the first media content; determine a target position based on the plurality of reference positions; and determine a second image from the first image that matches the frame information based on the target position, wherein the target position corresponds to the center point of the second image.

[0128] In some embodiments, the apparatus 400 further includes a weight determination module, configured to: determine weight information corresponding to multiple feature points based on the feature point types corresponding to the multiple feature points; and determine the target location based on the multiple feature points and the weight information.

[0129] In some embodiments, the feature point type includes at least one of the following: a first feature point type associated with an object of a preset type in the first image; a second feature point type associated with media material in the first media content; a third feature point type associated with interactive information of the first media content; and a fourth feature point type associated with the screen content of the first image.

[0130] In some embodiments, the second image retains an image region corresponding to a preset object in the first image.

[0131] In some embodiments, the apparatus 400 further includes a first material determination module configured to: determine a first material of a first type from a first group of media materials, the first type indicating that the first material is associated with target screen content in a first image; and add the first material to a second group of media materials in response to the target screen content being retained in a second image.

[0132] In some embodiments, the device 400 further includes an adjustment module configured to: adjust the display position of the first material based on the transformation relationship between the first image and the second image, so that the first material is displayed at the target screen content in the second image.

[0133] In some embodiments, the device 400 further includes an application module configured to: the first type of media material includes special effects stickers or pendants applied to preset objects in the image.

[0134] In some embodiments, the device 400 further includes a second material determination module, configured to: determine a second material of a second type from a first group of media materials, the second type indicating that the second material has picture content independent of the first image; and add the second material to the second group of media materials in response to the second material meeting preset conditions.

[0135] In some embodiments, the apparatus 400 further includes a display position determination module configured to determine the display position of the second material in the second media content based on the relative position of the second material with respect to the first media content.

[0136] In some embodiments, preset conditions indicate that the second material is a sticker or pendant that supports interactive operations.

[0137] In some embodiments, the display size of at least one of the media materials in the second group is determined based on the scaling ratio of the first image and the second image.

[0138] In some embodiments, the trigger module 430 is further configured to trigger the display of second media content in a target media component corresponding to the frame information.

[0139] In some embodiments, the target media component includes at least one of the following: a widget that matches the frame information, a camera component, or a media display component.

[0140] As shown in Figure 5, the electronic device 500 is in the form of a general-purpose electronic device. Components of the electronic device 500 may include, but are not limited to, one or more processing units or processors 510, memory 520, storage devices 530, one or more communication units 540, one or more input devices 550, and one or more output devices 560. The processor 510 may be a physical or virtual processor and is capable of performing various processes according to the programs stored in the memory 520. In a multiprocessor system, multiple processors execute computer-executable instructions in parallel to improve the parallel processing capability of the electronic device 500.

[0141] Electronic device 500 typically includes multiple computer storage media. Such media can be any accessible media that is accessible to electronic device 500, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 520 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 530 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media that can be used to store information and / or data and can be accessed within electronic device 500.

[0142] Electronic device 500 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not shown in FIG. 5, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks may be provided. In these cases, each drive may be connected to a bus (not shown) via one or more data media interfaces. Memory 520 may include computer program product 525 having one or more program modules configured to perform various methods or actions of various embodiments of the present disclosure.

[0143] Communication unit 540 enables communication with other electronic devices via a communication medium. Additionally, the functionality of components of electronic device 500 can be implemented using a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, electronic device 500 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network node.

[0144] Input device 550 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 560 can be one or more output devices, such as a monitor, speaker, printer, etc. Electronic device 500 can also communicate with one or more external devices (not shown) via communication unit 540 as needed. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with electronic device 500, or with any device that enables electronic device 500 to communicate with one or more other electronic devices (e.g., network card, modem, etc.). Such communication can be performed via input / output (I / O) interface (not shown).

[0145] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. According to an exemplary implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above.

[0146] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0147] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0148] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0149] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0150] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method for processing media content, comprising: Receive a frame change instruction associated with first media content, the first media content including a first image and a first set of media materials; Acquire second media content, which includes a second image and a second set of media materials. The second image is determined from the first image based on the frame information indicated by the frame change instruction, and the second set of media materials is determined based on the type of the first set of media materials. as well as Trigger the display of the second media content.

2. The method according to claim 1, further comprising: Based on the feature information of the first media content, multiple reference positions in the first media content are determined; Based on the multiple reference locations, the target location is determined; as well as Based on the target location, a second image matching the frame information is determined from the first image, wherein the target location corresponds to the center point of the second image.

3. The method according to claim 2, wherein the reference positions include a plurality of feature points in the first media content, and determining the target position based on the plurality of reference positions includes: Based on the feature point types corresponding to the multiple feature points, determine the weight information corresponding to the multiple feature points; as well as The target location is determined based on the multiple feature points and the weight information.

4. The method according to claim 3, wherein the feature point type includes at least one of the following: The first feature point type is associated with an object of a preset type in the first image; The second feature point type is associated with the media material in the first media content; The third feature point type is associated with the interactive information of the first media content; The fourth feature point type is associated with the content of the first image.

5. The method according to any one of claims 1 to 4, wherein the second image retains an image region corresponding to a preset object in the first image.

6. The method according to any one of claims 1 to 5, further comprising: From the first group of media materials, a first material of a first type is determined, wherein the first type indicates that the first material is associated with the target image content in the first image; as well as In response to the target image content being retained in the second image, the first material is added to the second set of media materials.

7. The method according to claim 6, further comprising: Based on the transformation relationship between the first image and the second image, the display position of the first material is adjusted so that the first material is displayed in the target screen content of the second image.

8. The method according to claim 7, further comprising: The first type of media material includes special effects stickers or pendants applied to preset objects in the image.

9. The method according to any one of claims 1 to 5, further comprising: A second type of material is determined from the first group of media materials, wherein the second type indicates that the content of the second material is independent of the image content of the first image; as well as In response to the second material meeting the preset conditions, the second material is added to the second group of media materials.

10. The method of claim 9, further comprising: The display position of the second material within the second media content is determined based on the relative position of the second material with respect to the first media content.

11. The method of claim 9, wherein the preset condition indicates that the second material is a sticker or pendant that supports interactive operation.

12. The method according to any one of claims 1 to 11, wherein the display size of at least one of the media materials in the second group is determined based on the scaling ratio of the first image and the second image.

13. The method according to any one of claims 1 to 12, wherein triggering the display of the second media content comprises: This triggers the display of the second media content in the target media component corresponding to the image size information.

14. The method of claim 13, wherein the target media component comprises at least one of the following: a widget, a camera component, or a media display component.

15. An apparatus for processing media content, comprising: The receiving module is configured to receive a frame change instruction associated with first media content, the first media content including a first image and a first set of media materials; The acquisition module is configured to acquire second media content, which includes a second image and a second set of media materials. The second image is determined from the first image based on the frame information indicated by the frame change instruction, and the second set of media materials is determined based on the type of the first set of media materials. as well as The trigger module is configured to trigger the display of the second media content.

16. An electronic device comprising: At least one processor; as well as At least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor, the instructions causing the electronic device to perform the method according to any one of claims 1 to 14 when executed by the at least one processor.

17. A computer-readable storage medium having stored thereon computer-executable instructions that, when executed by a processor, implement the method according to any one of claims 1 to 14.

18. A computer program product tangibly stored in a computer storage medium and comprising computer-executable instructions that, when executed by a device, cause the device to perform the method according to any one of claims 1 to 14.

Images