Method, apparatus, device and storage medium for processing media content

Abstract

According to an embodiment of the present disclosure, a method, device, equipment and computer readable storage medium for processing media content are provided. The method comprises: receiving a frame changing instruction associated with first media content, the first media content comprising a first image and a first set of media materials; obtaining second media content, the second media content comprising a second image and a second set of media materials, the second image being determined from the first image based on frame information indicated by the frame changing instruction, and the second set of media materials being determined based on a type of the first set of media materials; and triggering display of the second media content. Thus, the embodiment of the present disclosure can improve the quality of the media content after frame changing.

Description

TECHNICAL FIELD

[0001] Example embodiments of the present disclosure generally relate to the field of computers, and in particular, to a method, apparatus, device and computer readable storage medium for processing media content. BACKGROUND

[0002] With the development of computer technology, the Internet has become an important platform for people to obtain various information. Media content is an important form of information bearing. Some platforms may support users to add some media materials (e.g., stickers, etc.) on an image or video file to create more rich media content, for example. In some scenarios, the frame of the media content can not match the display component, which leads to the need to crop the media content. SUMMARY

[0003] In a first aspect of the present disclosure, a method for processing media content is provided. The method comprises: receiving a frame change instruction associated with first media content, the first media content comprising a first image and a first set of media materials; obtaining second media content, the second media content comprising a second image and a second set of media materials, the second image being determined from the first image based on frame information indicated by the frame change instruction, the second set of media materials being determined based on a type of the first set of media materials; and triggering display of the second media content.

[0004] In a second aspect of the present disclosure, an apparatus for processing media content is provided. The apparatus comprises: a receiving module configured to receive a frame change instruction associated with first media content, the first media content comprising a first image and a first set of media materials; an obtaining module configured to obtain second media content, the second media content comprising a second image and a second set of media materials, the second image being determined from the first image based on frame information indicated by the frame change instruction, the second set of media materials being determined based on a type of the first set of media materials; and a triggering module configured to trigger display of the second media content.

[0005] In a third aspect of the present disclosure, an electronic device is provided. The device comprises at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. The instructions, when executed by the at least one processing unit, cause the device to perform the method of the first aspect.

[0006] In a fourth aspect of the present disclosure, a computer readable storage medium is provided. The computer readable storage medium has stored thereon a computer program, the computer program being executable by a processor to implement the method of the first aspect.

[0007] It is to be understood that the particulars shown herein are by way of example and for purposes of illustrative discussion of the embodiments of the present disclosure only and are not intended to limit the scope of the present disclosure to the particular embodiments illustrated. Other BRIEF DESCRIPTION OF DRAWINGS

[0008] The above-mentioned and other features and advantages of various embodiments of this disclosure will become more apparent by reference to the following detailed description taken in conjunction with the accompanying drawings. In the drawings, like reference numerals designate like elements, wherein:

[0009] FIG. 1 A schematic diagram showing an example environment in which embodiments according to the present disclosure can be implemented is shown;

[0010] FIGS. 2A-2F An example interface according to some embodiments of the present disclosure is shown;

[0011] FIG. 3 A flowchart showing an example process for the generation of a music video according to some embodiments of the present disclosure is shown;

[0012] FIG. 4 A schematic block diagram showing an example apparatus for the generation of a music video according to some embodiments of the present disclosure is shown; and

[0013] FIG. 5 A block diagram of an electronic device capable of implementing a number of embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0014] Embodiments of the present disclosure will be described below in greater detail with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be interpreted as being limited to the embodiments set forth herein; rather, these embodiments are provided so as to more completely and thoroughly understand the present disclosure. It is to be understood that the drawings and embodiments of the present disclosure are for exemplary purposes only and are not intended to limit the scope of protection of the present disclosure.

[0015] It is noted that the headings provided herein are not limitations of the various embodiments described herein. Various embodiments are described throughout this document and can be included under any heading. Additionally, embodiments described in any heading can be combined with any other embodiment described in the same heading and / or a different heading in any manner.

[0016] In the description of the embodiments of the disclosure, the term "comprising" and similar terms thereof are understood to encompass open-ended inclusion, i.e., "comprising but not limited to". The term "based on" is understood to mean "based at least in part on". The term "one embodiment" or "the embodiment" is understood to mean "at least one embodiment". The term "some embodiments" is understood to mean "at least some embodiments". The following can also include other explicit and implicit definitions. The terms "first", "second", etc. can refer to different or same objects. The following can also include other explicit and implicit definitions.

[0017] The embodiments of the present disclosure can involve data of users, acquisition and / or use of data, etc. These aspects all comply with corresponding laws and regulations and relevant provisions. In the embodiments of the present disclosure, all data collection, acquisition, processing, processing, forwarding, use, etc. are carried out on the premise that the user is aware of and confirms. Accordingly, when implementing the embodiments of the present disclosure, the type of data or information that can be involved, the use range, the use scenario, etc. should be informed to the user and the authorization of the user should be obtained through appropriate means according to relevant laws and regulations. The specific informing and / or authorization manner can vary according to the actual situation and application scenario, and the scope of the present disclosure is not limited in this aspect.

[0018] In the present specification and embodiments, if the scheme involves processing of personal information, the processing is carried out on the premise of having a legal basis (for example, obtaining the consent of the subject of personal information, or being necessary for the performance of a contract, etc.) and is carried out only within the prescribed or agreed range. The user refuses to process personal information other than the necessary information required for the basic function, which does not affect the user's use of the basic function.

[0019] As mentioned above, the frame of the media content can not match the display component, which leads to the need to crop the media content. The traditional cropping method is usually to directly crop the corresponding region according to the frame of the component, which can lead to the loss of key information in the media content, thereby greatly affecting the viewing experience of the media content.

[0020] Embodiments of the present disclosure provide a scheme for processing media content. The scheme includes: receiving a frame changing instruction associated with first media content, the first media content including a first image and a first set of media materials; obtaining second media content, the second media content including a second image and a second set of media materials, the second image being determined from the first image based on frame information indicated by the frame changing instruction, the second set of media materials being determined based on a type of the first set of media materials; and triggering display of the second media content.

[0021] In one aspect, embodiments of the present disclosure are capable of cropping images in the media content according to the frame information; in another aspect, embodiments of the present disclosure are also capable of determining the retention policy of the material according to the type of the material in the original media content. In this way, embodiments of the present disclosure are capable of improving the quality of the media content after frame changing.

[0022] Various example implementations of the solution are described in further detail below in conjunction with the accompanying drawings.

[0023] Example Environment

[0024] FIG. 1 A schematic diagram illustrating an example environment 100 in which embodiments of the present disclosure can be implemented is shown. As shown, the example environment 100 can include an electronic device 110. FIG. 1

[0025] In this example environment 100, the electronic device 110 can run an application 120 that supports the generation of music videos. The application 120 can be any suitable type of application, examples of which can include, but are not limited to, a video application, a live broadcast application, or other suitable application capable of providing music video generation services. A user 140 can interact with the application 120 via the electronic device 110 and / or its attached devices.

[0026] In the example environment 100, if the application 120 is in an active state, the electronic device 110 can present an interface 150 for supporting the generation of music videos through the application 120. FIG. 1

[0027] In some embodiments, the electronic device 110 communicates with a server 130 to implement the provision of services to the application 120. The electronic device 110 can be any type of mobile terminal, fixed terminal, or portable terminal including a mobile handset, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a media computer, a multimedia tablet, a palmtop computer, a portable gaming terminal, a VR / AR device, a Personal Communication System (PCS) device, a personal navigation device, a Personal Digital Assistant (PDA), an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio broadcast receiver, an electronic book device, a game device, or any combination thereof, including accessories and peripherals of these devices, or any combination thereof. In some embodiments, the electronic device 110 can also support any type of interface to the user (such as "wearable" circuitry, etc.).

[0028] ​​The server 130 can be a standalone physical server, a server cluster composed of multiple physical servers, or a distributed system, and can also be a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content distribution networks, and basic cloud computing services such as big data and artificial intelligence platforms. The server 130 can include, for example, a computing system / server such as a mainframe, an edge computing node, a computing device in a cloud environment, and the like. The server 130 can provide background services for the application 120 in the electronic device 110 that supports the generation of music videos.

[0029] A communication connection can be established between the server 130 and the electronic device 110. The communication connection can be established by wired or wireless means. The communication connection can include, but is not limited to, a Bluetooth connection, a mobile network connection, a Universal Serial Bus (USB) connection, a Wireless Fidelity (WiFi) connection, and the like, and embodiments of the present disclosure are not limited in this regard. In embodiments of the present disclosure, the server 130 and the electronic device 110 can achieve signaling interaction through the communication connection therebetween.

[0030] It should be understood that the structure and function of the various elements in the environment 100 are described for illustrative purposes only, without implying any limitation on the scope of the present disclosure.

[0031] Some example embodiments of the present disclosure will be described below with continued reference to the accompanying drawings.

[0032] Example Interaction

[0033] FIG. 2A A first media content 205 according to some embodiments of the present disclosure is shown. As shown, the first media content 205 can include a first image 210 and a set of media materials, e.g., media material 220 and media material 225. As an example, the first media content 205 corresponds to a first frame, e.g., 4:3, for example. FIG. 2A

[0034] In some embodiments, the first image 210 can include, for example, a photo, a video, or an image captured in real time in a viewfinder.

[0035] In addition, the media material 220 and the media material 225 can be displayed, for example, superimposed on the first image 210. As an example, such media material 220 and media material 225 can be added based on user editing operations. For example, the media material 220 and the media material 225 can include materials generated based on special effects applied by the user.

[0036] ​In some embodiments, the media material can also include stickers for presenting information in the media content. For example, such stickers can include, but are not limited to, time stickers, weather stickers, magnifying glass stickers, text stickers, lyrics stickers, etc.

[0037] Additionally, the media material can also include a pendant. As an example, the pendant can have a corresponding functional attribute, such as a voting pendant, a video reply pendant, a point of interest pendant, a live broadcast preview pendant, a video forwarding pendant, etc. In some scenarios, the pendant can also be understood as a special type of sticker.

[0038] In some scenarios, such stickers and pendants are interactive. For example, a sticker or a pendant can be configured with corresponding interaction logic. For example, a user can trigger access to other interfaces or perform a predetermined interaction action by clicking on a sticker or a pendant in the first media content 205.

[0039] Taking the voting pendant as an example, a user can perform a corresponding voting action by clicking on the voting pendant in the first media content 205, for example.

[0040] In some embodiments, the media material can also include other appropriate types of interface components, for example. Such interface components can be configured and constitute part of the first media content 205 together with the first image 210.

[0041] As mentioned above, frame changing of media content is a common scenario. As an example, when the display component for presenting the media content does not match the current frame of the media content, the frame of the media content needs to be adjusted.

[0042] Taking the voting pendant as an example, a user can perform a corresponding voting action by clicking on the voting pendant in the first media content 205, for example. FIG. 2B As an example, the interface 200B can include a widget 230, for example. As an example, such a widget 230 can be associated with the application 120 and support displaying information associated with the application 120. In some scenarios, the widget 230 can be displayed on the system desktop, which generally has a fixed frame, for example. As an example, the frame corresponding to the widget 230 is 1:1.

[0043] In some scenarios, the widget 230 needs to adjust the frame of the first media content 205 when it is configured to display the first media content 205. For example, the first media content 205 can be a widget associated with a user in the application 120, which can display the works published by the user, for example. As an example, when the user publishes the first media content 205, the widget 230 needs to display the second media content 235 obtained by adjusting the frame of the first media content 205.

[0044] As introduced above, the conventional aspect ratio adjustment is usually based on the center point cropping manner, which results in the image content and media material being possibly displayed in a split manner, thus seriously affecting the information acquisition efficiency. For example, in the case that the first image 210 includes a human face, the center cropping manner can result in the user face being not completely displayed.

[0045] As shown in FIG. 2, the embodiments of the present disclosure can consider the content of the first image 210 and the type of the media material in the process of aspect ratio adjustment, and can accordingly obtain the second media content 235 matching the target aspect ratio (e.g., 1:1). FIG. 2B

[0046] In an aspect, the second media content 235 can include a second image obtained by performing aspect ratio cropping on the first image 210. As an example, the second image can correspond to the aspect ratio (e.g., 1:1) supported by the widget 230.

[0047] On the other hand, the second media content can also retain at least part of the media material (also referred to as a second group of media material) in the first media content 205. As an example, the media material 220 and the media material 225 (e.g., the hat sticker) can be retained.

[0048] As will be described in detail below, the retained second group of media material can be determined based on the type of the media material in the first media content 205. As an example, if the first media content 205 includes an interactive type of pendant, the second media content 235 can retain the interactive type of pendant regardless of whether it is retained within the cropping range.

[0049] By considering the type of the media material in the process of aspect ratio adjustment, the embodiments of the present disclosure can improve the quality of the media content after aspect ratio change, so as to improve the completeness of information display.

[0050] In addition to the widget scenario shown in FIG. 2, the embodiments of the present disclosure are also applicable to other appropriate aspect ratio adjustment scenarios. FIG. 2B For example, in the process of media content editing, a user can select the first media content 205 in a first aspect ratio. Further, the user can initiate an aspect ratio adjustment request for the first media content 205, so as to adjust the aspect ratio of the first media content 205 from 4:3 to 1:1, for example.

[0051]

[0052] ​​As another example, in a scenario of publishing media content, a user can select a first media content 205 of a first frame. Further, according to the type or platform of the published work, the frame of the first media content 205 can be automatically adjusted, or a plurality of preset frames can be provided to support user selection. For example, certain specific types of works only support a 1: 1 frame, so when the user publishes this type of work with the first media content 205, the frame of the first media content 205 can be automatically adjusted.

[0053] As yet another example, in a scenario of framing with a camera, a user can obtain, for example, an image captured in real time by the camera. In some scenarios, the user can also adjust the capture frame of the camera, so that the frame can be changed correspondingly.

[0054] In yet some examples, some specific media display components have a pre-set frame. For example, in a browsing interface of an information stream, the frame of the media display component of the media content can be pre-fixed. When the media content to be played does not match the frame of the media display component, the frame of the media content also needs to be adjusted.

[0055] The specific process of frame adjustment will be described in detail below in conjunction with FIGS. 2C-2F It should be understood that such a frame adjustment process can be implemented at a suitable electronic device, for example, the server 130 or the electronic device 110 displaying the media content after the frame change.

[0056] In some embodiments, the electronic device 110 and / or the server 130 can receive a frame change instruction associated with the first media content 205. Such a frame change instruction can be automatically generated by the system, or can also be generated based on user interaction.

[0057] Further, the electronic device 110 and / or the server 130 can determine the cropping range in the first image 210 of the first media content 205 based on the frame information indicated by the frame change instruction.

[0058] In some embodiments, the electronic device 110 and / or the server 130 can determine the feature information of the first media content 205, and can accordingly determine a plurality of reference positions in the first media content 205.

[0059] FIG. 2C A plurality of example reference positions according to some embodiments of the present disclosure are shown. As FIG. 2CAs shown, electronic device 110 and / or server 130 can acquire feature information associated with the first media content 205. In some embodiments, the feature information may be related to the screen content of the first image 210. For example, the feature information may indicate a preset object in the first image 210.

[0060] Alternatively or additionally, the feature information may be associated with a first set of media materials in the first media content. For example, the feature information may indicate the image location corresponding to the first set of media materials.

[0061] Alternatively or additionally, the feature information may also be related to the interactive information of the first media content 205. For example, special effects information may indicate the historical visual focus position of the first media content 205.

[0062] Specifically, electronic device 110 and / or server 130 can identify one or more feature points in the first media content 205 based on feature information of the first media content 205. For example... FIG. 2C As shown, electronic device 110 and / or server 130 can identify multiple feature points, namely, feature points 240-1 to feature points 240-5 (referred to individually or collectively as feature point 240).

[0063] In some embodiments, depending on how the feature points 240 are determined, such feature points may be associated with different feature point types. Furthermore, different types of feature point types may correspond to different weights.

[0064] As an example, the feature point type may include a first feature point type, which may be associated with an object of a predetermined type in the first image. As an example, the electronic device 110 and / or the server 130 may identify whether the first image includes an object of a predetermined type (e.g., a facial object), and may, for example, determine the location corresponding to the facial object as a first type of feature point.

[0065] In some examples, the weights corresponding to the first type of feature points can also be associated with the type of object. For example, a facial object may have the highest weight. Furthermore, objects such as food, clothing, and buildings may have lower weights than facial objects.

[0066] by FIG. 2C As an example, feature point 240-2 may correspond to a first type of feature point, which may, for example, correspond to the center point of a facial object in the first image.

[0067] In some embodiments, the feature point types can include a second feature point type, which is associated with media material in the first media content 205. As an example, the electronic device 110 and / or the server 130 can determine locations of media material in the first media content 205 as the second type of feature points. For example, the feature point 240-1, the feature point 240-3, and the feature point 240-4 can correspond to the second type of feature points, which can correspond to locations of media material such as stickers or hanging objects in the image.

[0068] In some examples, the weight of the second type of feature points can be lower than the weight of the first type of feature points, for example.

[0069] In some embodiments, the feature point types can include a third feature point type, which can be associated with interaction information of the first media content 205. As an example, the electronic device 110 and / or the server 130 can determine historical visual focus locations of the first media content 205 as the third type of feature points. Alternatively, the electronic device 110 and / or the server 130 can also determine a center location of the first media content 205 as the third type of feature points.

[0070] In addition, the electronic device 110 and / or the server 130 can also determine a visual dwell duration of the first media content 205, and can determine the weight of the third type of feature points based on the visual dwell duration. For example, the greater the visual dwell duration, the higher the weight of the third type of feature points, for example.

[0071] Additionally or alternatively, the feature point types can also include a fourth feature point type, which can be associated with picture content of the first image. For example, the electronic device 110 and / or the server 130 can determine regions in the first image with strong visual contrast and / or regions with high information density, and can determine fourth type of feature points corresponding to these regions.

[0072] As an example, the electronic device 110 and / or the server 130 can determine an image region in the first image with strong picture contrast and high saturation, and can determine a center point of the region as the fourth type of feature point. As yet another example, the electronic device 110 and / or the server 130 can also determine a region in the image with a large number of entity objects. For example, the electronic device 110 and / or the server 130 can determine a region in the first image with a large number of entity objects (e.g., clouds), and can determine a center point of the region as the fourth type of feature point, i.e., the feature point 240-5. FIG. 2C As an example, the electronic device 110 and / or the server 130 can determine a bottom region of the first image with a large number of entity objects (e.g., clouds), and can determine a center point of the region as the fourth type of feature point, i.e., the feature point 240-5.

[0073] In some embodiments, the number of feature points of each type can be set with a predetermined upper limit. When the number of feature points of a certain type exceeds the upper limit, the electronic device 110 and / or the server can randomly sample the feature points of the type, for example.

[0074] Further, the electronic device 110 and / or the server 130 can determine the target position based on the plurality of feature points 240 and the corresponding weight information. As an example, the electronic device 110 and / or the server 130 can determine a weighted barycenter position of the plurality of feature points 240 as the target position.

[0075] As shown in FIG. 2B, the electronic device 110 and / or the server 130 can determine a target position 245 based on the plurality of feature points 240, for example. In some embodiments, the target position 245 can be determined as a center point of the picture region to be cropped. In this way, embodiments of the present disclosure can ensure that important picture content of the first media content can be preserved in the cropping region. FIG. 2D

[0076] Further, the electronic device 110 and / or the server 130 can determine a cropping region 250 based on the target position 245 and a picture size (e.g., 1:1) to be cropped, and in turn can determine a second image matching the picture size from the first image 210.

[0077] Specifically, the electronic device 110 and / or the server 130 can take the target position 245 as a center point, for example, and scale the cropping region based on the picture size (e.g., 1:1) to be cropped, until the cropping region is the same as the length or width of the first image.

[0078] In this way, the electronic device 110 and / or the server 130 can implement adaptive picture cropping based on picture content of the first image, so as to be able to preserve picture regions with higher weights.

[0079] In some embodiments, in the process of determining the cropping region, the electronic device 110 and / or the server 130 can also check whether the cropping region completely preserves an image region corresponding to a preset object in the first image. For example, if the cropping region only preserves a partial region of a face object in the first image, the electronic device 110 and / or the server 130 can discard the cropping region, and can re-determine the target position.

[0080] As an example, in the process of re-determining the target position, the electronic device 110 and / or the server 130 can re-sample feature points of each type to re-determine a center point of the cropping region. As another example, in the process of re-determining the target position, the electronic device 110 and / or the server 130 can not consider feature points with reduced weights, or further reduce the weights of these feature points, for example, to preferentially ensure that feature points with higher weights can be preserved.

[0081] ​After the cropping area is determined, the electronic device 110 and / or the server 130 can accordingly acquire the second image to be retained. The dynamic adaptation of media material during aspect ratio changes will be further described below.

[0082] In some embodiments, the electronic device 110 and / or the server 130 may implement different adaptation strategies for different types of media materials.

[0083] As an example, media materials can be divided into information display materials and interactive materials. Further, information display materials can be further divided into media materials associated with the visual content and media materials independent of the visual content.

[0084] by FIG. 2E As an example, media material 225 may include sticker material generated based on special effects, such as a hat. This sticker material may belong to the information display category. Furthermore, the sticker material is associated with the content of the image; that is, the sticker material is attached to the head position in the image.

[0085] In this case, if the target image content (e.g., head) corresponding to media material 225 is retained in the second image, then media material 225 will be retained in the second media content, regardless of whether the cropping area 250 completely covers media material 225.

[0086] In addition, the electronic device 110 and / or the server 130 can also adjust the display position of the media material 225 based on the transformation relationship between the first image and the second image, so that the media material 225 is displayed at the target screen content (e.g., the head) in the second image.

[0087] In some embodiments, the electronic device 110 and / or the server 130 may determine a transformation matrix between the first image and the second image, and may determine the display position of the media material 225 in the second image based on the transformation matrix.

[0088] As an example, the transformation matrix can be represented as:

[0089]

[0090] Where Scale X represents the scaling ratio in the X direction, Scale Y represents the scaling ratio in the Y direction, Skew X represents the skew in the X direction, Skew Y represents the skew in the Y direction, Translate X represents the translation of the center point in the X direction, and Translate Y represents the translation of the center point in the Y direction.

[0091] Conversely, if the target screen content corresponding to the information display material is not retained in the second image, then the information display material may also not be retained. For example, if the original media content also includes sticker material corresponding to the hand position, and the hand position is not retained in the second image, then the sticker material corresponding to the hand position may also not be retained.

[0092] In some embodiments, the electronic device 110 and / or the server 130 may also determine a second piece of content, independent of the first image, from the first set of media materials. Further, the electronic device 110 and / or the server 130 may determine whether the second piece of content meets preset conditions.

[0093] Furthermore, if the second material is information display material unrelated to the image content, the electronic device 110 and / or server 130 can determine whether it needs to be retained based on the cropping area 250. For example, if media material 220 is unrelated to the image content in the first image and is retained in the cropping area 250, it can continue to be retained in the second media content.

[0094] Furthermore, if the second material is interactive (e.g., interactive stickers or widgets), the electronic device 110 and / or server 130 may retain it in the second media content.

[0095] In some embodiments, for interactive materials, the electronic device 110 and / or server 130 may retain the relative position of the interactive materials within the media content. Specifically, the electronic device 110 and / or server 130 may determine the relative position of the interactive materials with respect to the first media content 205, and determine the display position of the interactive materials within the second media content.

[0096] by FIG. 2F As an example, the first media content may include media material 255. As an example, the media material may include a challenge widget 255, which users can click to participate in a corresponding challenge interaction.

[0097] Challenge widget 255, for example, can always be retained within secondary media content, regardless of whether its corresponding position is within the cropping area. FIG. 2F As an example, Challenge Plugin 255 can be partially cropped. In this case, Challenge Plugin 255 can still be added to the cropped secondary media content.

[0098] In some embodiments, during the adjustment of the aspect ratio, the electronic device 110 and / or the server 130 can also maintain the relative position of the interactive type of media material. For example, the electronic device 110 and / or the server 130 can determine the distance of the challenge pendant 255 from the boundary of the first media content to determine its relative position. As an example, the electronic device 110 and / or the server 130 can determine the relative position of the voting pendant as (1 / 3*X, 1 / 4*Y). The relative distance can represent, for example, that its distance to the left boundary is one third of the width of the first media content and its distance to the top boundary is one fourth of the height of the first media content.

[0099] Accordingly, the electronic device 110 and / or the server 130 can maintain the relative position unchanged and can display it at the relative position (1 / 3*X, 1 / 4*Y) in the second media content, thereby maintaining the consistency of the relative position of the challenge pendant 255.

[0100] In some embodiments, the electronic device 110 and / or the server 130 can also adaptively adjust the size of the media material. Specifically, the electronic device 110 and / or the server 130 can determine the display size of the media material 225 based on the scaling ratio of the first image and the second image.

[0101] As an example, during the adjustment of the aspect ratio of the first media content from 4:3 to 1:1, the size of the media material can be determined to be scaled by 3 / 4. Accordingly, the size of the media material that is maintained to be displayed in the second media content can also be adjusted to be, for example, 3 / 4 of the size in the first media content, thereby improving the degree of adaptation of the picture.

[0102] In some embodiments, for the media material associated with the picture content, the electronic device 110 can also dynamically determine the scaling ratio of the media material based on the scaling ratio of the corresponding picture content. As an example, the media material 225 can be associated with the head region in the first media content. If the adjustment of the aspect ratio does not change the size of the head region, the size of the media material can not be scaled, for example.

[0103] In some embodiments, for the media material associated with the picture content, the electronic device 110 can also dynamically determine the scaling ratio so that it can be completely displayed in the second media content. Alternatively or additionally, for the media material that is completely within the cropping region, the electronic device 110 can not scale it, for example, while adaptively scaling the media material that is not completely within the cropping region and needs to be maintained.

[0104] In this way, embodiments of the present disclosure can achieve dynamic adaptation of media materials (such as stickers, pendants) during the frame adjustment process, thereby improving the quality of media content after frame adjustment and avoiding situations such as stickers or pendants being displayed in a fragmented manner or important pendants not being displayed.

[0105] Example Process

[0106] FIG. 3 A flowchart of an example process 300 for processing media content according to some embodiments of the present disclosure is shown. Process 300 can be implemented at electronic device 110. Reference is made below. FIG. 1 To describe process 300.

[0107] like FIG. 3 As shown, in box 310, electronic device 110 receives a frame change instruction associated with first media content, which includes a first image and a first set of media materials.

[0108] In frame 320, electronic device 110 acquires second media content, which includes a second image and a second set of media materials. The second image is determined from the first image based on the frame information indicated by the frame change instruction, and the second set of media materials is determined based on the type of the first set of media materials.

[0109] In frame 330, electronic device 110 triggers the display of second media content.

[0110] In some embodiments, process 300 further includes: determining a plurality of reference positions in the first media content based on feature information of the first media content; determining a target position based on the plurality of reference positions; and determining a second image from the first image that matches the frame information based on the target position, wherein the target position corresponds to the center point of the second image.

[0111] In some embodiments, the reference locations include multiple feature points in the first media content, and determining the target location based on the multiple reference locations includes: determining weight information corresponding to the multiple feature points based on the feature point types corresponding to the multiple feature points; and determining the target location based on the multiple feature points and the weight information.

[0112] In some embodiments, the feature point type includes at least one of the following: a first feature point type associated with an object of a preset type in the first image; a second feature point type associated with media material in the first media content; a third feature point type associated with interactive information of the first media content; and a fourth feature point type associated with the screen content of the first image.

[0113] In some embodiments, the second image retains an image region corresponding to a preset object in the first image.

[0114] In some embodiments, the process 300 further includes: determining, from the first set of media materials, a first material of a first type, the first type indicating that the first material is associated with the target frame content in the first image; and adding the first material to the second set of media materials in response to the target frame content being reserved in the second image.

[0115] In some embodiments, the process 300 further includes: adjusting a display position of the first material based on a transformation relationship between the first image and the second image, such that the first material is displayed at the target frame content in the second image.

[0116] In some embodiments, the process 300 further includes: the first type of media material includes a special effect sticker or a pendant applied to a preset object in the frame.

[0117] In some embodiments, the process 300 further includes: determining, from the first set of media materials, a second material of a second type, the second type indicating that the second material is independent of frame content of the first image; and adding the second material to the second set of media materials in response to the second material satisfying a preset condition.

[0118] In some embodiments, the process 300 further includes: determining a display position of the second material in the second media content based on a relative position of the second material relative to the first media content.

[0119] In some embodiments, the preset condition indicates that the second material is a sticker or a pendant supporting interactive operation.

[0120] In some embodiments, a display size of at least one material in the second set of media materials is determined based on a scaling ratio of the first image and the second image.

[0121] In some embodiments, triggering display of the second media content includes: triggering display of the second media content in a target media component corresponding to the frame information.

[0122] In some embodiments, the target media component includes at least one of: a widget matching the frame information, a camera component, or a media display component.

[0123] Example Devices and Equipment

[0124] Embodiments of the present disclosure also provide a corresponding apparatus for implementing the above method or process. FIG. 4 A schematic structural block diagram of an example apparatus 400 for processing media content according to certain embodiments of the present disclosure is shown. The apparatus 400 can be implemented as or included in the electronic device 110. Various modules / components in the apparatus 400 can be implemented by hardware, software, firmware, or any combination thereof.

[0125] As shown in FIG. 4 The apparatus 400 includes a receiving module 410 configured to receive a frame change instruction associated with a first media content, the first media content including a first image and a first set of media materials; an obtaining module 420 configured to obtain a second media content, the second media content including a second image and a second set of media materials, the second image being determined from the first image based on frame information indicated by the frame change instruction, the second set of media materials being determined based on a type of the first set of media materials; and a triggering module 430 configured to trigger display of the second media content.

[0126] In some embodiments, the apparatus 400 further includes a determining module configured to determine a plurality of reference positions in the first media content based on feature information of the first media content; determine a target position based on the plurality of reference positions; and determine the second image matching the frame information from the first image based on the target position, the target position corresponding to a center point of the second image.

[0127] In some embodiments, the apparatus 400 further includes a weight determining module configured to determine weight information corresponding to a plurality of feature points based on a type of the feature points corresponding to the plurality of feature points; and determine the target position based on the plurality of feature points and the weight information.

[0128] In some embodiments, the type of the feature points includes at least one of: a first type of feature points associated with a preset type of objects in the first image; a second type of feature points associated with media materials in the first media content; a third type of feature points associated with interaction information of the first media content; and a fourth type of feature points associated with a screen content of the first image.

[0129] In some embodiments, the second image retains an image region corresponding to a preset object in the first image.

[0130] In some embodiments, the apparatus 400 further includes a first material determining module configured to determine a first material of a first type from the first set of media materials, the first type indicating that the first material is associated with a target screen content in the first image; and add the first material to the second set of media materials in response to the target screen content being retained in the second image.

[0131] In some embodiments, the apparatus 400 further includes an adjusting module configured to adjust a display position of the first material based on a transformation relationship between the first image and the second image, so that the first material is displayed at the target screen content in the second image.

[0132] In some embodiments, the apparatus 400 further includes an applying module configured to the media material of the first type includes a special effect sticker or a pendant applied to a preset object in the screen.

[0133] In some embodiments, the device 400 further includes a second material determination module, configured to: determine a second material of a second type from a first group of media materials, the second type indicating that the second material has picture content independent of the first image; and add the second material to the second group of media materials in response to the second material meeting preset conditions.

[0134] In some embodiments, the apparatus 400 further includes a display position determination module configured to determine the display position of the second material in the second media content based on the relative position of the second material with respect to the first media content.

[0135] In some embodiments, preset conditions indicate that the second material is a sticker or pendant that supports interactive operations.

[0136] In some embodiments, the display size of at least one of the media materials in the second group is determined based on the scaling ratio of the first image and the second image.

[0137] In some embodiments, the trigger module 430 is further configured to trigger the display of second media content in a target media component corresponding to the frame information.

[0138] In some embodiments, the target media component includes at least one of the following: a widget that matches the frame information, a camera component, or a media display component.

[0139] like FIG. 5 As shown, electronic device 500 is in the form of a general-purpose electronic device. Components of electronic device 500 may include, but are not limited to, one or more processors or processing units 510, memory 520, storage device 530, one or more communication units 540, one or more input devices 550, and one or more output devices 560. Processing unit 510 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 520. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of electronic device 500.

[0140] The electronic device 500 typically includes a plurality of computer storage media. Such media can be any available media that is accessible by the electronic device 500 and includes both volatile and nonvolatile media, removable and non-removable media. The memory 520 can be volatile (such as register, cache, RAM), non-volatile (such as ROM, EEPROM, flash memory), or some combination of the two. The storage device 530 can be a removable or non-removable media, and can include machine-readable media, such as flash drives, magnetic disks, or any other media that can be used to store information and / or data and that can be accessed by the electronic device 500.

[0141] The electronic device 500 can further include additional removable / non-removable, volatile / non-volatile storage media. Although not shown in FIG. 5 FIG. 5, a disk drive for reading from or writing to a removable, non- volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive for reading from or writing to a removable, non-volatile optical disk (e.g., a CD-ROM) can be provided. In such instances, each drive can be connected to the bus (not shown) by one or more data media interfaces. The memory 520 can include a computer program product 525 having one or more program modules configured to carry out the various methods or actions of the various embodiments of the present disclosure.

[0142] The communication unit 540 enables communication with other electronic devices over a communication medium. Additionally, the functionality of the components of the electronic device 500 can be implemented in a single computing cluster or a plurality of computer machines capable of communicating over a communication connection. As such, the electronic device 500 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network nodes in the networking environment.

[0143] The input device 550 can be one or more input devices, such as a mouse, a keyboard, a trackball, etc. The output device 560 can be one or more output devices, such as a display, a speaker, a printer, etc. The electronic device 500 can also communicate with one or more external devices (not shown) such as a storage device, a display device, etc. through the communication unit 540, as needed, one or more devices that enable a user to interact with the electronic device 500, or any devices (e.g., a network card, a modem, etc.) that enable the electronic device 500 to communicate with one or more other electronic devices. Such communication can be carried out via an input / output (I / O) interface (not shown).

[0144] According to an example implementation of the present disclosure, a computer readable storage medium is provided having computer executable instructions stored thereon, where the computer executable instructions are executed by a processor to implement the method described above. According to an example implementation of the present disclosure, a computer program product is also provided that is tangibly stored on a non-transitory computer readable medium and includes computer executable instructions, where the computer executable instructions are executed by a processor to implement the method described above.

[0145] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0146] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0147] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0148] The computer program product of the present disclosure can have a signal including said computer program. This signal can be electronic, electromagnetic, optical, or any other suitable type of signal. Such a signal can be provided through a communication connection, such as electrical wiring, optical fiber, wireless interface, etc. Examples of computer program products include computer program implemented on a personal computer, server, or other networked device. A non-transitory computer readable medium, such as a floppy disk, CD-ROM, DVD-ROM, Blu-ray Disc, hard disk drive, or any other suitable non-transitory computer readable medium can store the computer program product.

[0149] Various implementations of the disclosure have been described in detail above. The foregoing description is exemplary and explanatory only, and is not intended to be exhaustive or to limit various implementations of the disclosure to the precise forms disclosed. Many modifications and variations are possible in light of the above teachings without departing from the scope and spirit of the disclosure. It is intended that the scope of the disclosure be limited only by the claims and the equivalents thereof. The use of the terms "including," "containing," "comprising," "having," "in involving," "portions," "elements," "components," "steps," "phases," "processes," "operations," "steps," "stages," "procedures," "methods," "mechanisms," "devices," "systems," "apparatuses," "units," "means," "units," "devices," "systems," "apparatuses," "units," "devices," "systems," "apparatuses," "units," "devices," "systems," "apparatuses," "units," "devices," "systems," "apparatuses," "units," "devices," "systems," "apparatuses," "units," "devices," "systems," "apparatuses," "units," "devices," "systems," "apparatuses," "units," "devices," "systems," "apparatuses," "units," "devices," "systems," "apparatuses," "units," "devices," "systems," "apparatuses," "units," "devices," "systems," "apparatuses," "units," "devices," "systems," "apparatuses," "units," "devices," "systems," "apparatuses," "units," "devices," "systems," "apparatuses," "units," "devices," "systems," "apparatuses," "units," "devices," "systems," "apparatuses," "units," "devices," "systems," "apparatuses," "units," "devices," "systems," "apparatuses," "units," "devices," "systems," "apparatuses," "units," "devices," "systems," "apparatuses," "units," "devices," "systems," "apparatuses," "units," "devices," "systems," "apparatuses," "units," "devices," "systems," "apparatuses," "units," "devices," "systems," "apparatuses," "units," "devices," "systems," "apparatuses," "units," "devices," "systems," "apparatuses," "units," "devices," "systems," "apparatuses," "units," "devices," "systems," "apparatuses," "units," "devices," "systems," "apparatuses," "units," "devices," "systems," "

Claims

1. A method for processing media content, comprising: receiving a frame change instruction associated with a first media content, the first media content comprising a first image and a first set of media materials; acquiring a second media content, the second media content comprising a second image and a second set of media materials, the second image being determined from the first image based on frame information indicated by the frame change instruction, the second set of media materials being determined from the first set of media materials based on types of the first set of media materials; and triggering display of the second media content.

2. The method of claim 1, further comprising: determining a plurality of reference positions in the first media content based on feature information of the first media content; determining a target position based on the plurality of reference positions; and determining the second image matching the frame information from the first image based on the target position, the target position corresponding to a center point of the second image.

3. The method of claim 2, wherein the reference positions comprise a plurality of feature points in the first media content, and determining a target position based on the plurality of reference positions comprises: determining weight information corresponding to the plurality of feature points based on types of the feature points corresponding to the plurality of feature points; and determining the target position based on the plurality of feature points and the weight information.

4. The method of claim 3, wherein the types of the feature points comprise at least one of: a first type of feature points associated with a preset type of objects in the first image; a second type of feature points associated with media materials in the first media content; a third type of feature points associated with interaction information of the first media content; a fourth type of feature points associated with a screen content of the first image.

5. The method of claim 1, wherein the second image retains an image region corresponding to a preset object in the first image.

6. The method of claim 1, further comprising: determining a first material of a first type from the first set of media materials, the first type indicating that the first material is associated with a target screen content in the first image; and adding the first material to the second set of media materials in response to the target screen content being retained in the second image.

7. The method of claim 6, further comprising: adjusting a display position of the first material based on a transformation relationship between the first image and the second image, such that the first material is displayed at the target screen content in the second image.

8. The method of claim 7, further comprising: the media material of the first type comprising a special effect sticker or a hanging object applied to a preset object in a screen.

9. The method of claim 1, further comprising: determining a second material of a second type from the first set of media materials, the second type indicating that the second material is independent of a screen content of the first image; and adding the second material to the second set of media materials in response to the second material satisfying a preset condition. ​ ​ ​ ​ ​ 10.The method of claim 9, further comprising: determining a display position of the second media content in the second media content based on a relative position of the second media material with respect to the first media content. 11.The method of claim 9, wherein the preset condition indicates that the second media material is a sticker or a pendant that supports interactive operation. 12.The method of claim 1, wherein a display size of at least one media material in the second set of media materials is determined based on a zoom scale of the first image and the second image. 13.The method of claim 1, wherein triggering display of the second media content comprises: triggering display of the second media content in a target media component corresponding to the frame information. 14.The method of claim 13, wherein the target media component comprises at least one of the following: a widget, a camera component, or a media showcase component. 15.An apparatus for processing media content, comprising: a receiving module configured to receive a frame change instruction associated with a first media content, the first media content comprising a first image and a first set of media materials; an obtaining module configured to obtain a second media content, the second media content comprising a second image and a second set of media materials, the second image being determined from the first image based on frame information indicated by the frame change instruction, the second set of media materials being determined from the first set of media materials based on a type of the first set of media materials; and a triggering module configured to trigger display of the second media content. 16.An electronic device, comprising: at least one processing unit; and at least one memory that is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, cause the electronic device to perform the method according to any one of claims 1-14. 17.A computer readable storage medium having stored thereon a computer program, the computer program, when executed by a processor, implements the method according to any one of claims 1-14. ​ ​

Images