Media content processing method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a media content processing method and device, equipment and a computer readable storage medium. The method comprises the following steps: receiving a picture change instruction associated with first media content, wherein the first media content comprises a first image and a first group of media materials; second media content is acquired, the second media content comprises a second image and a second group of media materials, the second image is determined from the first image based on the picture information indicated by the picture change instruction, and the second group of media materials is determined based on the type of the first group of media materials; and triggering to display the second media content. Therefore, the quality of the media content after picture change can be improved.

Description

Technical Field

[0001] Example embodiments of the present disclosure generally relate to the field of computers, and more particularly, to methods, devices, apparatuses, and computer-readable storage media for processing media content. Background Art

[0002] With the development of computer technology, the Internet has become an important platform for people to obtain various information. Media content is an important form of information carrier. Some platforms, for example, can support users to add some media materials (such as stickers, etc.) to images or video files to create richer media content. In some scenarios, the frame of the media content may not match the display component, which requires the media content to be cropped. Summary of the invention

[0003] In a first aspect of the present disclosure, a method for processing media content is provided. The method includes: receiving an aspect ratio change instruction associated with first media content, the first media content including a first image and a first group of media materials; acquiring second media content, the second media content including a second image and a second group of media materials, the second image being determined from the first image based on aspect ratio information indicated by the aspect ratio change instruction, the second group of media materials being determined based on the type of the first group of media materials; and triggering display of the second media content.

[0004] In a second aspect of the present disclosure, a device for processing media content is provided. The device includes: a receiving module configured to receive an aspect ratio change instruction associated with first media content, the first media content including a first image and a first group of media materials; an acquisition module configured to acquire second media content, the second media content including a second image and a second group of media materials, the second image is determined from the first image based on aspect ratio information indicated by the aspect ratio change instruction, and the second group of media materials is determined based on the type of the first group of media materials; and a triggering module configured to trigger display of the second media content.

[0005] In a third aspect of the present disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit. When the instructions are executed by the at least one processing unit, the device executes the method of the first aspect.

[0006] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the computer-readable storage medium, and the computer program can be executed by a processor to implement the method of the first aspect.

[0007] It should be understood that the contents described in this content section are not intended to limit the key features or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0009] Figure 1 A schematic diagram showing an example environment in which embodiments according to the present disclosure may be implemented;

[0010] FIG. 2A to FIG. 2F shows an example interface according to some embodiments of the present disclosure;

[0011] Figure 3 A flowchart illustrating an example process for the generation of a music video according to some embodiments of the present disclosure;

[0012] Figure 4 A schematic structural block diagram showing an example apparatus for generating a music video according to some embodiments of the present disclosure; and

[0013] Figure 5 A block diagram of an electronic device capable of implementing various embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0014] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0015] It should be noted that the titles of any sections / subsections provided herein are not restrictive. Various embodiments are described throughout this article, and any type of embodiment may be included under any section / subsection. In addition, the embodiments described in any section / subsection may be combined in any manner with any other embodiments described in the same section / subsection and / or different sections / subsections.

[0016] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may be included below. The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may be included below.

[0017] The embodiments of the present disclosure may involve user data, data acquisition and / or use, etc. These aspects are subject to the corresponding laws, regulations and relevant provisions. In the embodiments of the present disclosure, all data collection, acquisition, processing, processing, forwarding, use, etc. are carried out on the premise that the user knows and confirms. Accordingly, when implementing each embodiment of the present disclosure, the type, scope of use, usage scenario, etc. of the data or information that may be involved should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with the relevant laws and regulations. The specific notification and / or authorization method can vary according to the actual situation and application scenario, and the scope of the present disclosure is not limited in this respect.

[0018] In this specification and the embodiments, if personal information processing is involved, it will be processed on the premise of having a legal basis (such as obtaining the consent of the subject of personal information, or it is necessary to perform a contract, etc.), and will only be processed within the scope of regulations or agreements. If a user refuses to process personal information other than the necessary information for basic functions, it will not affect the user's use of basic functions.

[0019] As mentioned above, the frame of the media content may not match the display component, which requires the media content to be cropped. The traditional cropping method is usually to directly crop the corresponding area according to the frame of the component, which will cause the loss of key information in the media content, thus greatly affecting the viewing experience of the media content.

[0020] The embodiment of the present disclosure proposes a generation scheme. The scheme includes: receiving an aspect ratio change instruction associated with first media content, the first media content includes a first image and a first group of media materials; acquiring second media content, the second media content includes a second image and a second group of media materials, the second image is determined from the first image based on aspect ratio information indicated by the aspect ratio change instruction, and the second group of media materials is determined based on the type of the first group of media materials; and triggering the display of the second media content.

[0021] On the one hand, the embodiments of the present disclosure can crop images in media content according to the frame information; on the other hand, the embodiments of the present disclosure can also determine the material retention strategy according to the type of material in the original media content. In this way, the embodiments of the present disclosure can improve the quality of the media content after the frame is changed.

[0022] Various example implementations of the solution are described in detail below in conjunction with the accompanying drawings.

[0023] Example Environment

[0024] Figure 1 1 is a schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented. Figure 1 As shown, example environment 100 may include electronic device 110 .

[0025] In this example environment 100, the electronic device 110 may run an application 120 that supports the generation of a music video. The application 120 may be any suitable type of application, examples of which may include, but are not limited to, video applications, live broadcast applications, or other suitable applications that can provide music video generation services. The user 140 may interact with the application 120 via the electronic device 110 and / or its attached devices.

[0026] exist Figure 1 In the environment 100 , if the application 120 is active, the electronic device 110 may present an interface 150 for supporting generation of a music video through the application 120 .

[0027] In some embodiments, the electronic device 110 communicates with the server 130 to provide services for the application 120. The electronic device 110 can be any type of mobile terminal, fixed terminal or portable terminal, including a mobile phone, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a media computer, a multimedia tablet, a handheld computer, a portable game terminal, a VR / AR device, a personal communication system (PCS) device, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio receiver, an e-book device, a game device, or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof. In some embodiments, the electronic device 110 can also support any type of interface for the user (such as a "wearable" circuit, etc.).

[0028] The server 130 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks, and big data and artificial intelligence platforms. The server 130 may include, for example, a computing system / server, such as a mainframe, an edge computing node, a computing device in a cloud environment, etc. The server 130 may provide background services for the application 120 that supports the generation of music videos in the electronic device 110.

[0029] A communication connection may be established between the server 130 and the electronic device 110. The communication connection may be established in a wired manner or a wireless manner. The communication connection may include, but is not limited to, a Bluetooth connection, a mobile network connection, a Universal Serial Bus (USB) connection, a Wireless Fidelity (WiFi) connection, etc., and the embodiments of the present disclosure are not limited in this respect. In the embodiments of the present disclosure, the server 130 and the electronic device 110 may implement signaling interaction through the communication connection between the two.

[0030] It should be understood that the structure and function of the various elements in the environment 100 are described for exemplary purposes only and do not imply any limitation on the scope of the present disclosure.

[0031] Some example embodiments of the present disclosure will be described below with continued reference to the accompanying drawings.

[0032] Example Interaction

[0033] Figure 2A 2 shows first media content 205 according to some embodiments of the present disclosure. Figure 2A As shown, the first media content 205 may include a first image 210 and a group of media materials, for example, media material 220 and media material 225. As an example, the first media content 205 corresponds to a first frame, for example, 4:3.

[0034] In some embodiments, the first image 210 may include, for example, a photo, a video, or an image captured in real time in a viewfinder.

[0035] In addition, media material 220 and media material 225 may be displayed superimposed on first image 210, for example. As an example, such media material 220 and media material 225 may be added based on the editing operation of the user. For example, media material 220 and media material 225 may include materials generated based on special effects applied by the user.

[0036] In some embodiments, the media material may also include stickers for presenting information in the media content. For example, such stickers may include but are not limited to: time stickers, weather stickers, magnifying glass stickers, text stickers, lyrics stickers, etc.

[0037] Additionally, the media material may also include widgets. As an example, a widget may have corresponding functional attributes, such as a voting widget, a video reply widget, a point of interest widget, a live broadcast preview widget, a video forwarding widget, etc. In some scenarios, a widget may also be understood as a special type of sticker.

[0038] In some scenarios, such stickers and widgets are interactive. For example, stickers or widgets may be configured with corresponding interactive logic. For example, a user may trigger access to other interfaces or perform a predetermined interactive action by clicking on a sticker or widget in the first media content 205.

[0039] Taking a voting widget as an example, a user may perform a corresponding voting action by, for example, clicking on the voting widget in the first media content 205 .

[0040] In some embodiments, the media material may further include, for example, other appropriate types of interface components. Such interface components may be configured to constitute a part of the first media content 205 together with the first image 210 .

[0041] As mentioned above, changing the frame size of media content is a common scenario. As an example, when the display component used to present the media content does not match the current frame size of the media content, the frame size of the media content needs to be adjusted.

[0042] by Figure 2B As an example, the interface 200B may include a widget 230. As an example, such a widget 230 may be associated with the application 120 and also support displaying information associated with the application 120. In some scenarios, the widget 230 may be displayed on the system desktop, which, for example, usually has a fixed frame. As an example, the frame corresponding to the widget 230 is 1:1.

[0043] In some scenarios, when the widget 230 is configured to display the first media content 205, it is necessary to adjust the frame of the first media content 205. For example, the first media content 205 may be a widget associated with a user in the application 120, which may display works published by the user. As an example, when the user publishes the first media content 205, the widget 230 needs to display the second media content 235 obtained by adjusting the frame of the first media content 205.

[0044] As described above, conventional frame adjustment is usually based on a center point cropping method, which may cause the image content and media materials to be displayed separately, thereby seriously affecting the efficiency of information acquisition. For example, when the first image 210 includes a person's face, the middle cropping method may cause the user's face to be unable to be fully displayed.

[0045] like Figure 2B As shown, the embodiments of the present disclosure may consider the content of the first image 210 and the type of media material during the frame adjustment process, and may accordingly obtain the second media content 235 matching the target frame (eg, 1:1).

[0046] In one aspect, the second media content 235 may include a second image obtained by cropping the first image 210. As an example, the second image may correspond to an aspect ratio (eg, 1:1) supported by the widget 230.

[0047] On the other hand, the second media content may also retain at least part of the media material (also referred to as a second set of media materials) in the first media content 205. As an example, the media material 220 and the media material 225 (eg, a hat sticker) may be retained.

[0048] As will be described in detail below, the retained second group of media materials may be determined based on the type of media materials in the first media content 205. As an example, if the first media content 205 includes an interactive widget, the second media content 230 may retain the interactive widget regardless of whether it is retained within the cropping range.

[0049] By considering the type of media material during the frame adjustment process, the embodiments of the present disclosure can improve the quality of the media content after the frame change, so as to enhance the integrity of information display.

[0050] remove Figure 2B In addition to the widget scenario shown, the embodiments of the present disclosure are also applicable to other appropriate frame adjustment scenarios.

[0051] For example, during the media content editing process, the user may select the first media content 205 of the first frame. Further, the user may initiate a canvas adjustment request for the first media content 205, for example, to adjust the frame of the first media content 205 from 4:3 to 1:1.

[0052] As another example, in the scenario of publishing media content, the user can select the first media content 205 of the first frame. Further, according to the type or platform of the published work, the frame of the first media content 205 can be automatically adjusted, or multiple preset frames can be provided to support user selection. For example, some specific types of works only support a 1:1 frame, so when the user uses the first media content 205 to publish the type of work, the frame of the first media content 205 can be automatically adjusted.

[0053] As another example, in a scene where a camera is used for framing, the user can obtain an image captured by the camera in real time. In some scenes, the user can also adjust the capture frame of the camera, so that the frame can be changed accordingly.

[0054] In some other examples, some specific media display components have a preset frame size. For example, in the browsing interface of the information stream, the frame size of the media display component of the media content can be fixed in advance. When the media content to be played does not match the frame size of the media display component, the frame size of the media content also needs to be adjusted.

[0055] The following will be combined Figure 2C to Figure 2F It should be understood that such an adjustment process can be implemented in a suitable electronic device, such as the server 130 or the electronic device 110 that displays the media content after the canvas is changed.

[0056] In some embodiments, the electronic device 110 and / or the server 130 may receive an image change instruction associated with the first media content 205. Such an image change instruction may be automatically generated by the system, or may also be generated based on a user's interactive operation.

[0057] Furthermore, the electronic device 110 and / or the server 130 may determine a cropping range in the first image 210 of the first media content 205 based on the frame information indicated by the frame change instruction.

[0058] In some embodiments, the electronic device 110 and / or the server 130 may determine feature information of the first media content 205 , and may accordingly determine a plurality of reference locations in the first media content 205 .

[0059] Figure 2C Several example reference positions according to some embodiments of the present disclosure are shown. Figure 2CAs shown, the electronic device 110 and / or the server 130 may obtain feature information associated with the first media content 205. In some embodiments, the feature information may be related to the screen content of the first image 210. For example, the feature information may indicate a preset object in the first image 210.

[0060] Alternatively or additionally, the characteristic information may be related to the first group of media materials in the first media content. For example, the characteristic information may indicate the image position corresponding to the first group of media materials.

[0061] Alternatively or additionally, the feature information may also be related to the interaction information of the first media content 205. For example, the special effect information may indicate the historical visual focus position of the first media content 205.

[0062] Specifically, the electronic device 110 and / or the server 130 may identify one or more feature points in the first media content 205 based on the feature information of the first media content 205. Figure 2C As shown, the electronic device 110 and / or the server 130 may identify a plurality of feature points, namely, feature points 204 - 1 to 204 - 5 (individually or collectively referred to as feature points 204 ).

[0063] In some embodiments, depending on the manner in which the feature points 204 are determined, such feature points may be associated with different feature point types. In addition, different types of feature point types may also correspond to different weights.

[0064] As an example, the feature point type may include a first feature point type, which may be associated with a preset type of object in the first image. As an example, the electronic device 110 and / or the server 130 may identify whether the first image includes a predetermined type of object (e.g., a facial object), and may, for example, determine a position corresponding to the facial object as a first type of feature point.

[0065] In some examples, the weight corresponding to the first type of feature point may also be associated with the type of object. For example, a facial object may have the highest weight. In addition, the weight of objects such as food, clothing, and buildings may be lower than the weight of a facial object.

[0066] by Figure 2C As an example, the feature point 204 - 2 may correspond to a first type of feature point, which may, for example, correspond to a center point of a facial object of the first image.

[0067] In some embodiments, the feature point type may include a second feature point type, which is associated with a media material in the first media content 205. As an example, the electronic device 110 and / or the server 130 may determine the position corresponding to each media material in the first media content 205 as a second type of feature point. For example, feature point 240-1, feature point 240-3, and feature point 240-4 may correspond to the second type of feature point, which may correspond to the position of a media material such as a sticker or pendant in an image.

[0068] In some examples, the weight of the second type of feature points may be lower than the weight of the first type of feature points, for example.

[0069] In some embodiments, the feature point type may include a third feature point type, which may be associated with the interaction information of the first media content 205. As an example, the electronic device 110 and / or the server 130 may determine the historical visual focus position of the first media content 205 as the third type of feature point. Alternatively, the electronic device 110 and / or the server 130 may also use the center position of the first media content 205 as the third type of feature point.

[0070] In addition, the electronic device 110 and / or the server 130 may also determine the visual dwell time of the first media content 205, and may determine the weight of the third type of feature point based on the visual dwell time. For example, the greater the visual dwell time, the higher the weight of the third type of feature point may be.

[0071] Additionally or alternatively, the feature point type may further include a fourth feature point type, which may be associated with the screen content of the first image. For example, the electronic device 110 and / or the server 130 may determine areas with strong visual contrast and / or areas with high information density in the first image, and may determine fourth-type feature points corresponding to these areas.

[0072] As an example, the electronic device 110 and / or the server 130 may determine an image region with strong contrast and high saturation in the first image, and may determine the center point of the region as a fourth type of feature point. As another example, the electronic device 110 and / or the server 130 may also determine an area with a large number of physical objects in the image. Figure 2C As an example, the electronic device 110 and / or the server 130 may determine that the bottom area of ​​the first image has a densely packed physical object (eg, clouds), and may determine the center point of the area as a fourth type of feature point, ie, feature point 240 - 5 .

[0073] In some embodiments, the number of feature points of each type may be set with a predetermined upper limit, for example. When the number of features of a certain type exceeds the upper limit, the electronic device 110 and / or the server may randomly sample feature points of the type, for example.

[0074] Further, the electronic device 110 and / or the server 130 may determine the target location based on the plurality of feature points 240 and the corresponding weight information. As an example, the electronic device 110 and / or the server 130 may determine the weighted centroid location of the plurality of feature points 240 as the target location.

[0075] like Figure 2D As shown, the electronic device 110 and / or the server 130 can, for example, determine a target position 245 based on a plurality of feature points 240. In some embodiments, the target position 245 can be determined as the center point of the image area to be cropped. In this way, the embodiments of the present disclosure can ensure that the important image content of the first media content can be retained in the cropping area.

[0076] Furthermore, the electronic device 110 and / or the server 130 may determine the cropping area 250 based on the target position 245 and the frame to be cropped (eg, 1:1), and may further determine a second image matching the frame from the first image 210 .

[0077] Specifically, the electronic device 110 and / or the server 130 may, for example, use the target position 245 as the center point and scale the cropping area based on the frame to be cropped (eg, 1:1) until the cropping area is the same length or width as the first image.

[0078] Thus, the electronic device 110 and / or the server 130 can implement adaptive picture cropping based on the picture content of the first image, so as to retain the picture area with a higher weight.

[0079] In some embodiments, in the process of determining the cropping area, the electronic device 110 and / or the server 130 may also check whether the cropping area completely retains the image area corresponding to the preset object in the first image. For example, if the cropping area only retains a partial area of ​​the facial object in the first image, the electronic device 110 and / or the server 130 may abandon the cropping area and may re-determine the target position.

[0080] As an example, in the process of re-determining the target position, the electronic device 110 and / or the server 130 may resample various types of feature points to re-determine the center point of the cropped area. As another example, in the process of re-determining the target position, the electronic device 110 and / or the server 130 may, for example, not consider feature points with reduced weights, or further reduce the weights of these feature points to prioritize and ensure that feature points with higher weights can be retained.

[0081] After the cropping area is determined, the electronic device 110 and / or the server 130 may correspondingly obtain the second image to be retained. The following will further introduce the dynamic adaptation of the media material during the frame change process.

[0082] In some embodiments, the electronic device 110 and / or the server 130 may implement different adaptation strategies for different types of media materials.

[0083] As an example, media materials can be divided into information display materials and interactive materials. Furthermore, information display materials can be further divided into media materials associated with screen content and media materials independent of screen content.

[0084] by Figure 2E As an example, the media material 225 may include a sticker material generated based on special effects, such as a hat. The sticker material may belong to information display material. In addition, the sticker material has an association relationship with the image content. That is, the sticker material is bound to the head position in the image.

[0085] In this case, if the target image content (eg, the head) corresponding to the media material 225 remains in the second image, the media material 225 will be retained in the second media content regardless of whether the cropping area 250 completely covers the media material 225 .

[0086] In addition, the electronic device 110 and / or the server 130 may also adjust the display position of the media material 225 based on the transformation relationship between the first image and the second image so that the media material 225 is displayed at the target screen content (eg, the head) in the second image.

[0087] In some embodiments, the electronic device 110 and / or the server 130 may determine a transformation matrix between the first image and the second image, and may determine a display position of the media material 225 in the second image based on the transformation matrix.

[0088] As an example, the transformation matrix can be expressed as:

[0089]

[0090] Among them, Scale X represents the scaling ratio in the X direction, Scale Y represents the scaling ratio in the Y direction, Skew X represents the skew in the X direction, Skew Y represents the skew in the Y direction, Translate X represents the translation of the center point in the X direction, and Translate Y represents the translation of the center point in the Y direction.

[0091] On the contrary, if the target screen content corresponding to the information display material is not retained in the second image, the information display material may not be retained. For example, if the original media content also includes sticker material corresponding to the hand position, and the hand position is not retained in the second image, the sticker material corresponding to the hand position may not be retained.

[0092] In some embodiments, the electronic device 110 and / or the server 130 may also determine a second material independent of the screen content of the first image from the first set of media materials. Further, the electronic device 110 and / or the server 130 may determine whether the second material meets a preset condition.

[0093] In addition, if the second material is an information display material that is not related to the screen content, the electronic device 110 and / or the server 130 can determine whether it needs to be retained based on the cropping area 250. For example, if the media material 220 is irrelevant to the screen content in the first image and is retained in the cropping area 250, it can continue to be retained in the second media content.

[0094] In addition, if the second material is an interactive material (for example, a sticker or pendant supporting interaction), the electronic device 110 and / or the server 130 may retain it in the second media content.

[0095] In some embodiments, for interactive material, the electronic device 110 and / or the server 130 may retain the relative position of the interactive material in the media content. Specifically, the electronic device 110 and / or the server 130 may determine the relative position of the interactive material relative to the first media content 205, and determine the display position of the interactive material in the second media content.

[0096] by Figure 2F As an example, the first media content may include, for example, a media material 255. As an example, the media material may include, for example, a challenge widget 255, and the user may participate in the corresponding challenge interaction by, for example, clicking on the challenge widget 255.

[0097] The challenge widget 255 may, for example, always be retained in the second media content, regardless of whether its corresponding position is within the cropping area. Figure 2F As an example, the challenge widget 255 may be partially cut, for example. In this case, the challenge widget 255 may still be added to the cut second media content.

[0098] In some embodiments, during the process of adjusting the frame, the electronic device 110 and / or the server 130 may also maintain the relative position of the interactive material. For example, the electronic device 110 and / or the server 130 may determine the distance between the challenge widget 255 and the border of the first media content to determine its relative position. As an example, the electronic device 110 and / or the server 130 may determine the relative position of the voting widget to be (1 / 3*X, 1 / 4*Y). This relative distance may, for example, indicate that the distance to the left border is one-third of the width of the first media content, and the distance to the upper border is one-quarter of the height of the first media content.

[0099] Accordingly, the electronic device 110 and / or the server 130 may keep its relative position unchanged and may display it at the relative position (1 / 3*X, 1 / 4*Y) in the second media content, thereby keeping the consistency of the relative position of the challenge widget 255 .

[0100] In some embodiments, the electronic device 110 and / or the server 130 may also adaptively adjust the size of the media material. Specifically, the electronic device 110 and / or the server 130 may determine the display size of the media material 225 based on the scaling ratio of the first image and the second image.

[0101] As an example, when the aspect ratio of the first media content is changed from 4:3 to 1:1, its size can be considered to be scaled by 3 / 4. Accordingly, the size of the media material retained and displayed in the second media content can also be adjusted to 3 / 4 of the size in the first media content, thereby improving the degree of picture adaptation.

[0102] In some embodiments, for media materials associated with the picture content, the electronic device 110 may also dynamically determine the scaling ratio of the media materials based on the scaling ratio of the corresponding picture content. As an example, the media material 225 may be associated with the head area in the first media content. If the frame adjustment process does not change the size of the head area, the size of the media material may not be scaled, for example.

[0103] In some embodiments, for the media material associated with the screen content, the electronic device 110 may also dynamically determine its scaling ratio so that it can be fully displayed in the second media content. Alternatively or additionally, for the media material that falls completely within the cropping area, the electronic device 110 may, for example, not scale it, but adaptively scale the media material that does not fall completely within the cropping area and needs to be retained.

[0104] In this way, the embodiments of the present disclosure can achieve dynamic adaptation of media materials (such as stickers and pendants) during the frame adjustment process, thereby improving the quality of the media content after the frame adjustment and avoiding situations such as stickers or pendants being displayed in pieces or important pendants not being displayed.

[0105] Example Process

[0106] Figure 3 FIG. 3 is a flowchart showing an example process 300 for processing media content according to some embodiments of the present disclosure. The process 300 may be implemented at the electronic device 110. Figure 1 The process 300 is described below.

[0107] like Figure 3 As shown, in box 310, the electronic device 110 receives an image change instruction associated with a first media content, where the first media content includes a first image and a first group of media materials.

[0108] In box 320, the electronic device 110 obtains second media content, which includes a second image and a second group of media materials. The second image is determined from the first image based on the frame information indicated by the frame change instruction, and the second group of media materials is determined based on the type of the first group of media materials.

[0109] At block 330 , the electronic device 110 triggers display of the second media content.

[0110] In some embodiments, process 300 also includes: determining multiple reference positions in the first media content based on feature information of the first media content; determining a target position based on the multiple reference positions; and determining a second image matching the frame information from the first image based on the target position, wherein the target position corresponds to a center point of the second image.

[0111] In some embodiments, the reference position includes multiple feature points in the first media content, and based on the multiple reference positions, determining the target position includes: determining weight information corresponding to the multiple feature points based on feature point types corresponding to the multiple feature points; and determining the target position based on the multiple feature points and the weight information.

[0112] In some embodiments, the feature point type includes at least one of the following: a first feature point type, associated with an object of a preset type in the first image; a second feature point type, associated with a media material in the first media content; a third feature point type, associated with interactive information of the first media content; and a fourth feature point type, associated with the picture content of the first image.

[0113] In some embodiments, the second image retains an image area corresponding to a preset object in the first image.

[0114] In some embodiments, process 300 further includes: determining a first material of a first type from the first set of media materials, the first type indicating that the first material is associated with target screen content in the first image; and in response to the target screen content being retained in the second image, adding the first material to the second set of media materials.

[0115] In some embodiments, the process 300 further includes: adjusting the display position of the first material based on the transformation relationship between the first image and the second image, so that the first material is displayed at the target screen content in the second image.

[0116] In some embodiments, the process 300 further includes: the first type of media material includes special effect stickers or pendants applied to a preset object in the picture.

[0117] In some embodiments, process 300 further includes: determining a second material of a second type from the first group of media materials, the second type indicating that the second material is independent of the screen content of the first image; and in response to the second material satisfying a preset condition, adding the second material to the second group of media materials.

[0118] In some embodiments, process 300 further includes determining a display position of the second material in the second media content based on a relative position of the second material with respect to the first media content.

[0119] In some embodiments, the preset condition indicates that the second material is a sticker or a pendant that supports interactive operations.

[0120] In some embodiments, a display size of at least one of the second set of media materials is determined based on a scaling ratio of the first image to the second image.

[0121] In some embodiments, triggering display of the second media content includes: triggering display of the second media content in a target media component corresponding to the frame information.

[0122] In some embodiments, the target media component includes at least one of the following: a widget, a camera component, or a media presentation component that matches the frame information.

[0123] Example devices and equipment

[0124] The embodiments of the present disclosure also provide corresponding devices for implementing the above methods or processes. Figure 4 A schematic structural block diagram of an example apparatus 400 for processing media content according to some embodiments of the present disclosure is shown. The apparatus 400 may be implemented as or included in the electronic device 110. Each module / component in the apparatus 400 may be implemented by hardware, software, firmware, or any combination thereof.

[0125] like Figure 4 As shown, the device 400 includes: a receiving module 410, configured to receive an aspect ratio change instruction associated with a first media content, the first media content includes a first image and a first group of media materials; an acquisition module 420, configured to acquire a second media content, the second media content includes a second image and a second group of media materials, the second image is determined from the first image based on the aspect ratio information indicated by the aspect ratio change instruction, and the second group of media materials is determined based on the type of the first group of media materials; and a triggering module 430, configured to trigger the display of the second media content.

[0126] In some embodiments, the device 400 also includes a determination module configured to: determine multiple reference positions in the first media content based on feature information of the first media content; determine a target position based on the multiple reference positions; and determine a second image matching the frame information from the first image based on the target position, wherein the target position corresponds to the center point of the second image.

[0127] In some embodiments, the device 400 also includes a weight determination module configured to: determine weight information corresponding to the multiple feature points based on the feature point types corresponding to the multiple feature points; and determine the target position based on the multiple feature points and the weight information.

[0128] In some embodiments, the feature point type includes at least one of the following: a first feature point type, associated with an object of a preset type in the first image; a second feature point type, associated with a media material in the first media content; a third feature point type, associated with interactive information of the first media content; and a fourth feature point type, associated with the picture content of the first image.

[0129] In some embodiments, the second image retains an image area corresponding to a preset object in the first image.

[0130] In some embodiments, the device 400 also includes a first material determination module, which is configured to: determine a first material of a first type from the first group of media materials, the first type indicating that the first material is associated with a target screen content in the first image; and in response to the target screen content being retained in the second image, add the first material to the second group of media materials.

[0131] In some embodiments, the apparatus 400 further includes an adjustment module configured to adjust a display position of the first material based on a transformation relationship between the first image and the second image, so that the first material is displayed at a target screen content in the second image.

[0132] In some embodiments, the apparatus 400 further includes an application module configured as follows: the first type of media material includes special effect stickers or pendants applied to a preset object in the picture.

[0133] In some embodiments, the device 400 also includes a second material determination module, which is configured to: determine a second material of a second type from the first group of media materials, the second type indicating that the second material is independent of the picture content of the first image; and in response to the second material satisfying a preset condition, add the second material to the second group of media materials.

[0134] In some embodiments, the apparatus 400 further includes a display position determination module configured to determine a display position of the second material in the second media content based on a relative position of the second material with respect to the first media content.

[0135] In some embodiments, the preset condition indicates that the second material is a sticker or a pendant that supports interactive operations.

[0136] In some embodiments, a display size of at least one of the second set of media materials is determined based on a scaling ratio of the first image to the second image.

[0137] In some embodiments, the trigger module 430 is further configured to: trigger display of the second media content in the target media component corresponding to the frame information.

[0138] In some embodiments, the target media component includes at least one of the following: a widget, a camera component, or a media presentation component that matches the frame information.

[0139] like Figure 5 As shown, the electronic device 500 is in the form of a general electronic device. The components of the electronic device 500 may include, but are not limited to, one or more processors or processing units 510, a memory 520, a storage device 530, one or more communication units 540, one or more input devices 550, and one or more output devices 560. The processing unit 510 may be an actual or virtual processor and is capable of performing various processes according to a program stored in the memory 520. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to improve the parallel processing capability of the electronic device 500.

[0140] The electronic device 500 typically includes a plurality of computer storage media. Such media may be any accessible media that is accessible to the electronic device 500, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 520 may be a volatile memory (e.g., registers, caches, random access memory (RAM)), a non-volatile memory (e.g., a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 530 may be a removable or non-removable medium, and may include a machine-readable medium, such as a flash drive, a disk, or any other medium, which may be capable of being used to store information and / or data and may be accessed within the electronic device 500.

[0141] The electronic device 500 may further include additional removable / non-removable, volatile / non-volatile storage media. Figure 5 As shown in , a disk drive for reading or writing from a removable, non-volatile disk (e.g., a "floppy disk") and an optical drive for reading or writing from a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to the bus (not shown) by one or more data media interfaces. The memory 520 may include a computer program product 525 having one or more program modules that are configured to perform various methods or actions of various embodiments of the present disclosure.

[0142] The communication unit 540 implements communication with other electronic devices through a communication medium. Additionally, the functions of the components of the electronic device 500 can be implemented in a single computing cluster or multiple computing machines that can communicate through a communication connection. Therefore, the electronic device 500 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.

[0143] The input device 550 may be one or more input devices, such as a mouse, a keyboard, a tracking ball, etc. The output device 560 may be one or more output devices, such as a display, a speaker, a printer, etc. The electronic device 500 may also communicate with one or more external devices (not shown) through the communication unit 540 as needed, such as a storage device, a display device, etc., communicate with one or more devices that allow a user to interact with the electronic device 500, or communicate with any device that allows the electronic device 500 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0144] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.

[0145] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices, equipment, and computer program products implemented according to the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0146] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0147] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0148] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple implementations of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some implementations as replacements, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0149] The above descriptions of various implementations of the present disclosure are exemplary, non-exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The selection of terms used herein is intended to best explain the principles of the implementations, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the various implementations disclosed herein.

Claims

1. A method for processing media content, comprising: Receiving an image frame change instruction associated with first media content, wherein the first media content includes a first image and a first set of media materials; Acquire second media content, the second media content including a second image and a second group of media materials, the second image is determined from the first image based on the frame information indicated by the frame change instruction, and the second group of media materials is determined based on the type of the first group of media materials; as well as The display of the second media content is triggered.

2. The method according to claim 1, further comprising: Determining a plurality of reference positions in the first media content based on the characteristic information of the first media content; determining a target position based on the plurality of reference positions; as well as Based on the target position, the second image matching the frame information is determined from the first image, and the target position corresponds to a center point of the second image.

3. The method according to claim 2, wherein the reference position comprises a plurality of feature points in the first media content, and based on the plurality of reference positions, determining the target position comprises: Determining weight information corresponding to the plurality of feature points based on the feature point types corresponding to the plurality of feature points; as well as The target position is determined based on the plurality of feature points and the weight information.

4. The method according to claim 3, wherein the feature point type comprises at least one of the following: a first feature point type associated with an object of a preset type in the first image; a second feature point type, associated with a media material in the first media content; a third feature point type, associated with the interaction information of the first media content; The fourth feature point type is associated with the picture content of the first image. The method according to claim 1 , wherein the second image retains an image area corresponding to a preset object in the first image.

6. The method according to claim 1, further comprising: Determining a first material of a first type from the first group of media materials, the first type indicating that the first material is associated with target screen content in the first image; as well as In response to the target screen content being retained in the second image, the first material is added to the second set of media materials.

7. The method according to claim 6, further comprising: Based on the transformation relationship between the first image and the second image, the display position of the first material is adjusted so that the first material is displayed at the target screen content in the second image.

8. The method according to claim 7, further comprising: The first type of media material includes special effect stickers or pendants applied to preset objects in the picture.

9. The method according to claim 1, further comprising: Determining a second material of a second type from the first group of media materials, the second type indicating that the second material is independent of picture content of the first image; as well as In response to the second material satisfying a preset condition, the second material is added to the second group of media materials.

10. The method according to claim 9, further comprising: Based on the relative position of the second material with respect to the first media content, a display position of the second material in the second media content is determined.

11. The method according to claim 9, wherein the preset condition indicates that the second material is a sticker or a pendant supporting interactive operations. 12 . The method of claim 1 , wherein a display size of at least one item in the second set of media materials is determined based on a scaling ratio of the first image and the second image.

13. The method of claim 1, wherein triggering display of the second media content comprises: Triggering display of the second media content in a target media component corresponding to the frame information. The method according to claim 13 , wherein the target media component comprises at least one of the following: a widget, a camera component, or a media presentation component.

15. An apparatus for processing media content, comprising: A receiving module is configured to receive an image frame change instruction associated with first media content, wherein the first media content includes a first image and a first group of media materials; an acquisition module, configured to acquire second media content, the second media content comprising a second image and a second group of media materials, the second image being determined from the first image based on the frame information indicated by the frame change instruction, and the second group of media materials being determined based on types of the first group of media materials; as well as The trigger module is configured to trigger display of the second media content.

16. An electronic device, comprising: at least one processing unit; as well as At least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to any one of claims 1 to 14 when executed by the at least one processing unit.

17. A computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the method according to any one of claims 1 to 14 when executed by a processor.

Images