A method, apparatus, device and storage medium for processing media content

Abstract

Embodiments of the present disclosure relate to a method, apparatus, electronic device and storage medium for processing media content. The method comprises: obtaining first media content; applying a target effect to the first media content using a target model to generate second media content; and providing the second media content, wherein the target model is trained based on the following process: training a pre-trained model associated with the target effect using a first sample set to determine a reference model; constructing a second sample set based on processing results of the reference model for a plurality of sample images; and training the target model using the second sample set. According to embodiments of the present disclosure, the quality of the second media content generated by the target model can be improved.

Description

Technical Field

[0001] The exemplary embodiments disclosed herein generally relate to the field of computers, and particularly to a method, apparatus, device, and computer-readable storage medium for processing media content. Background Technology

[0002] With the development of computer technology, terminal devices such as mobile phones have the ability to process media content in real time based on artificial intelligence technology.

[0003] However, due to the limitations of the terminal device's computing power, it may take a long time for the terminal device to process media content, which will affect the user experience. Summary of the Invention

[0004] In a first aspect of this disclosure, a method for processing media content is provided. The method includes: acquiring first media content; applying a target effect to the first media content using a target model to generate second media content; and providing the second media content, wherein the target model is trained based on the following process: training a pre-trained model associated with the target effect using a first sample set to determine a reference model; constructing a second sample set based on processing results of the reference model for multiple sample images; and training the target model using the second sample set.

[0005] In a second aspect of this disclosure, an apparatus for processing media content is provided. The apparatus includes: an acquisition module configured to acquire first media content; a processing module configured to apply a target effect to the first media content using a target model to generate second media content; and a providing module configured to provide the second media content, wherein the target model is trained based on the following process: training a pre-trained model associated with the target effect using a first sample set to determine a reference model; constructing a second sample set based on processing results of the reference model for multiple sample images; and training the target model using the second sample set.

[0006] In a third aspect of this disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. When executed by the at least one processing unit, the instructions cause the device to perform the method of the first aspect.

[0007] In a fourth aspect of this disclosure, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program that can be executed by a processor to implement the method of the first aspect.

[0008] It should be understood that the content described in this content section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0009] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0010] Figure 1 A schematic diagram is shown of an example environment in which embodiments of the present disclosure may be implemented;

[0011] Figure 2 A flowchart illustrating an example process for processing media content according to some embodiments of this disclosure is shown;

[0012] Figures 3A-3E Example interfaces according to some embodiments of this disclosure are shown;

[0013] Figure 4 A flowchart illustrating an example process for training a target model according to some embodiments of the present disclosure is shown;

[0014] Figure 5 A flowchart illustrating an example process for training a target model according to some embodiments of this disclosure is shown.

[0015] Figure 6 A flowchart illustrating an example process for constructing a second sample set according to some embodiments of this disclosure is shown.

[0016] Figure 7 A schematic diagram illustrating an example process for adjusting a first image according to some embodiments of the present disclosure is shown.

[0017] Figure 8 A schematic structural block diagram of an example apparatus for processing media content according to some embodiments of the present disclosure is shown; and

[0018] Figure 9 A block diagram of an electronic device capable of implementing several embodiments of the present disclosure is shown. Detailed Implementation

[0019] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0020] It should be noted that the headings of any section / subsection provided herein are not limiting. Various embodiments are described throughout this document, and embodiments of any type may be included under any section / subsection. Furthermore, embodiments described in any section / subsection may be combined in any way with any other embodiments described in the same section / subsection and / or different sections / subsections.

[0021] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below. The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0022] The embodiments of this disclosure may involve user data, data acquisition, and / or use. All of these aspects comply with applicable laws, regulations, and relevant provisions. In the embodiments of this disclosure, all data collection, acquisition, processing, manipulation, forwarding, and use are conducted with the user's knowledge and confirmation. Accordingly, in implementing the embodiments of this disclosure, the type, scope of use, and usage scenarios of any data or information that may be involved should be communicated to the user and their authorization obtained in accordance with relevant laws and regulations through appropriate means. The specific methods of notification and / or authorization may vary depending on the actual situation and application scenario, and the scope of this disclosure is not limited in this respect.

[0023] In this specification and the embodiments, any processing of personal information will be carried out only under the premise of legality (such as obtaining the consent of the personal information subject, or being necessary for the performance of a contract), and will only be carried out within the scope stipulated or agreed upon. A user's refusal to process personal information other than that necessary for basic functions will not affect the user's use of basic functions.

[0024] As mentioned above, terminal devices typically employ artificial intelligence models capable of processing media content. However, because these models have only been widely applied in real-world scenarios for a relatively short time, there is significant room for optimization, including but not limited to improvements in their learning capabilities and response speed. Once optimized, these models will take less time to process media content, thereby improving the user experience.

[0025] Embodiments of this disclosure propose a scheme for processing media content. The scheme includes: acquiring first media content; applying a target effect to the first media content using a target model to generate second media content; and providing the second media content, wherein the target model is trained based on the following process: training a pre-trained model associated with the target effect using a first sample set to determine a reference model; constructing a second sample set based on the processing results of the reference model for multiple sample images; and training the target model using the second sample set.

[0026] According to embodiments of this disclosure, by using a pre-trained model associated with the target effect to train a reference model to construct a second sample set for training the target model, embodiments of this disclosure can shorten the time required to train the target model, improve the training efficiency of the target model, and enhance the processing quality of the model.

[0027] The following section provides a detailed description of various example implementations of this scheme, with reference to the accompanying drawings.

[0028] Example Environment

[0029] Figure 1 A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. For example... Figure 1 As shown, example environment 100 may include terminal device 110.

[0030] In this example environment 100, terminal device 110 may run an application 120 that supports processing media content. Application 120 may be any suitable type of application for processing media content, examples of which may include, but are not limited to, image processing applications, video processing applications, or other suitable applications. User 140 may interact with application 120 via terminal device 110 and / or its attached devices.

[0031] exist Figure 1 In environment 100, if application 120 is active, terminal device 110 can use application 120 to present interface 150 for supporting the processing of media content.

[0032] In some embodiments, terminal device 110 communicates with server 130 to provide services to application 120. Terminal device 110 can be any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, handheld computers, portable gaming terminals, VR / AR devices, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio receivers, e-book devices, gaming devices, or any combination thereof, including accessories and peripherals of these devices or any combination thereof. In some embodiments, terminal device 110 can also support any type of user-facing interface (such as "wearable" circuitry).

[0033] Server 130 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms. Server 130 may include, for example, computing systems / servers such as mainframes, edge computing nodes, computing devices in a cloud environment, etc. Server 130 can provide backend services for applications 120 in terminal devices 110 that support media content processing.

[0034] A communication connection can be established between server 130 and terminal device 110. This communication connection can be established via wired or wireless means. The communication connection may include, but is not limited to, Bluetooth, mobile network, Universal Serial Bus (USB), and Wireless Fidelity (WiFi) connections; the embodiments of this disclosure are not limited in this respect. In the embodiments of this disclosure, server 130 and terminal device 110 can achieve signaling interaction through the communication connection between them.

[0035] It should be understood that the structure and function of the various elements in environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of this disclosure.

[0036] The following description will continue with reference to the accompanying drawings, which will provide some exemplary embodiments of this disclosure.

[0037] Example Process

[0038] Figure 2 A flowchart of an example process 200 for processing media content according to some embodiments of the present disclosure is shown. Process 200 can be implemented at terminal device 110. Reference is made below. Figure 1 To describe process 200.

[0039] like Figure 2 As shown in box 210, terminal device 110 acquires the first media content.

[0040] In some embodiments, the first media content may be media data obtained by the terminal device 110 from the user 140. The first media content may be presented in other forms such as images or videos. As an example, the first media content may be transmitted to the terminal device 110 by means of taking a picture, wired / wireless transmission, etc.

[0041] In some embodiments, such as Figure 3A As shown, the terminal device 110 can display an operation interface 300A configured for inputting first media content. The operation interface 300A may include, but is not limited to, buttons labeled "Upload" and "Take Photo".

[0042] When terminal device 110 receives user input to the "Upload" button, terminal device 110 can display interface 300B, such as... Figure 3B As shown. Interface 300B includes locally stored data such as a local photo album. Interface 300B also includes buttons for the user to select images, so that the terminal device 110 can upload the selected images.

[0043] When terminal device 110 receives user input to the "take photo" button, terminal device 110 invokes the camera function and displays the corresponding interface. For example... Figure 3C As shown, when the terminal device 110 acquires the shooting result, the terminal device 110 can present the shooting result through the interface 300C. As an example, the interface 300C may include, but is not limited to, an image preview area 310 indicating the shooting result, a button with the word "Select", and a button with the word "Retake Photo", so that the terminal device 110 can acquire the image obtained by shooting.

[0044] In some embodiments, such as Figure 3D As shown, after acquiring the image selected by the user, the terminal device 110 can display the interface 300D. As an example, the interface 300D can be configured with an image preview area 310 for the user to preview the selected image. Furthermore, the interface 300D can also be configured with a "Generate" control indicating the application of the corresponding effect to the target model, and can be configured with a button to return to the image selection step.

[0045] In box 220, terminal device 110 uses the target model to apply the target effect to the first media content to generate the second media content.

[0046] In some embodiments, the second media content is the media content formed after applying the target effect to the first media content. Similar to the presentation format of the first media content, the second media content can be in the form of an image, video, or other presentation formats.

[0047] In some embodiments, the target model can be of various types depending on the type of media content to be processed and the type of target effect. Examples may include, but are not limited to, models that can process portrait images, models that can process face images, and models that can process videos.

[0048] In frame 230, terminal device 110 provides second media content.

[0049] In some embodiments, such as Figure 3E As shown, after the terminal device 110 generates the second media content based on the first media content, the terminal device 110 can display information related to the second media content through the interface 300E to provide the second media content. As an example, the information related to the second media content may be at least one of a preview image of the second media content and a download link for the second media content.

[0050] It should be understood that Figures 3A-3E The media content generation interface shown is merely an example; other suitable interfaces can also be used to generate and provide second media content. The various graphic elements in the interface can have different arrangements and different visual representations, one or more elements can be omitted or replaced, and one or more other elements may also be present. The embodiments of this disclosure are not limited in this respect.

[0051] The following will be further combined Figure 4 and Figure 5 This describes the specific training process of the target model. Figure 4 A flowchart of an example process 400 for training a target model according to some embodiments of the present disclosure is shown. Figure 5 A flowchart of an example process 500 for training a target model according to some embodiments of the present disclosure is shown. It should be understood that processes 400 and / or 500 can be performed by suitable electronic devices, such as server 130. Process 600 will be described below using server 130 as an example.

[0052] like Figure 4 As shown in box 410, server 130 can use the first sample set to train a pre-trained model associated with the target effect to determine the reference model.

[0053] likeFigure 5 As shown, the first sample set 505 may include sample pairs associated with the target effect. Each sample pair may, for example, include an initial image and a target image to which the target effect is applied. In some embodiments, to improve training efficiency, the server 130 may determine a first pre-trained model 515 corresponding to the target goblin from a plurality of pre-trained models associated with different effects.

[0054] In some embodiments, the first pre-trained model 515 may include a machine learning model associated with the target effect, examples of which may include, but are not limited to, GANs (Generative Adversarial Networks). Taking makeup effects as an example, different makeup effects may be associated with different pre-trained GAN models.

[0055] In some embodiments, the processing effect achieved by the first pre-trained model 515 can be of the same type as the target effect. By training the first pre-trained model 515 using the first sample set 505, the time used to train the first pre-trained model 515 using the first sample set 505 can be saved. For example, if the target effect is a makeup effect A, the processing effect achieved by the first pre-trained model 515 can be another makeup effect B. As an example, the server 130 can select a model with a similar processing effect from multiple models that achieve the same effect as the first pre-trained model 515. Embodiments of this disclosure can reduce the training cost of the model and improve the training efficiency of the model by selecting a first pre-trained model 515 with an effect related to the target effect.

[0056] Additionally, when training the first pre-trained model 515 based on the first sample set 505, the server 130 can determine a training template 510 associated with the first pre-trained model 515 to further shorten the training time. In some instances, the training template 510 can be a combination of multiple hyperparameters. By using different training templates 510, embodiments of this disclosure effectively reduce the debugging process during model training and lower the training cost of the model.

[0057] In this way, server 130 can train first pre-trained model 515 using first sample set 505 to obtain reference model 525.

[0058] Continue to refer to Figure 2 In box 420, server 130 can construct a second sample set 545 based on the processing results of multiple sample images 520 by reference model 525.

[0059] In some embodiments, the sample image 520 may be a still image or a video image. In some scenarios, multiple sample images 520 may also simultaneously include real images and synthetic images. By combining real images and synthetic images, embodiments of this disclosure can improve the processing performance of the model.

[0060] In some embodiments, the second sample set 545 may include a plurality of sample images 520, and the processing result of the reference model 525 for each sample image. Each sample image 520 and the processing result of the reference model 525 for that sample image constitute a pair of paired data.

[0061] The following will be further combined Figure 6 This describes the specific process of constructing the second sample set 545. Figure 6 A flowchart of an example process 600 for constructing a second sample set 545 according to some embodiments of the present disclosure is shown.

[0062] refer to Figure 6 In box 610, server 130 uses reference model 525 to process multiple sample images 520 and generate multiple first images 530 corresponding to the multiple sample images 520.

[0063] In some embodiments, the first image 530 is the image result of the sample image 520 processed by the reference model 525. The type of the first image 530 is consistent with that of the sample image 520. For example, if multiple sample images 520 are portrait images, then the corresponding multiple first images 530 are also portrait images.

[0064] In box 620, server 130 can construct a second sample set 545 based on multiple first images 530.

[0065] In some embodiments, such as Figure 5 As shown, server 130 can also filter 535 and / or adjust 540 multiple first images 530 to obtain a higher quality second sample set 545.

[0066] In some embodiments, both the first image 530 and its corresponding sample image 520 may contain a preset object. The preset object can be the object to which the target effect is applied, and examples of such objects may be a person or an animal. As an example, when the processing effect of the reference model 525 is applied to the preset object, the processing effect may change the style of at least one feature point of the preset object. However, in practice, the processing effect of the reference model 525 may also change at least one feature point of the preset object. For example, the processing effect of the reference model 525 is a makeup effect. When the sample image 520 containing a human figure is processed using the reference model 525, the makeup effect changes the style of the eyelashes and eyebrows, as well as the position of the eyebrows. Here, the human figure in the sample image 520 is the preset object. The eyelashes and eyebrows are each a feature point of the preset object. The makeup effect changing the position of the eyebrows is equivalent to changing the feature point of the preset object. Thus, at 535, the server 130 can also obtain multiple second images by filtering at least one image that does not meet the preset conditions from multiple first images 530. Further, the server 130 can also construct a second sample set 545 based on the multiple second images and the corresponding sample images 520. In this way, embodiments of the present disclosure can improve the sample quality of the second sample set 545.

[0067] Specifically, server 130 may, for example, detect a set of feature points in each first image 530 that are associated with a preset object. Furthermore, server 130 may filter at least one image from the plurality of first images 530 that does not meet the preset conditions based on multiple sets of feature points.

[0068] In some embodiments, the preset conditions may be related to the number and / or positional relationship of a set of feature points in order to filter out images that are not suitable for applying the target effect. As an example, server 130 may filter one or more images from a plurality of first images 530 whose number of feature points is less than a threshold and / or whose positional relationship does not meet the preset conditions.

[0069] By filtering the first image 530, the server 130 can obtain multiple second images. The server 130 can further pair the multiple second images with corresponding sample images 520 to construct a second sample set 545.

[0070] Reference Figure 7In some embodiments, when the processing effect of the reference model 525 is applied to the sample image 520, the processing effect can change the preset effective range 710 of the target effect. However, in reality, the processing effect of the reference model 525 may also change other areas different from the preset effective range 710. For example, the processing effect of the reference model 525 is a makeup effect. When processing the sample image 520 containing a portrait using the reference model 525, the makeup effect adds a filter to the face area and changes the color of the person's arms and legs. Here, the preset effective range 710 of the makeup effect is the face area. The makeup effect changing the color of the person's arms and legs is equivalent to changing other areas different from the preset effective range 710. To this end, based on the preset effective range 710 of the target effect, a set of first image regions 720 independent of the target effect can be determined among the multiple first images 530. Then, based on the multiple sample images 520, the set of first image regions 720 among the multiple first images 530 is adjusted.

[0071] In some embodiments, a first image region 720 in the first image 530, independent of the target effect, may include a region in the first image 530 where the effect's effective range extends beyond a preset effective range 710. As an example, there may be multiple first image regions 720 in the first image 530. In this case, the multiple first image regions 720 may be referred to as a group of first image regions 720.

[0072] In some embodiments, server 130 may replace a set of first image regions 720 in first image 530 based on sample image 520. Alternatively or additionally, server 130 may also adjust attribute information of a set of first image regions 720 based on sample image 520.

[0073] In some embodiments, server 130 may determine a set of second image regions 730 on a sample image 520 associated with the first image 530, and may further replace a set of first image regions 720 with the set of second image regions 730.

[0074] like Figure 7 As shown, a set of second image regions 730 can be associated with a set of first image regions 720. As an example, the position of the set of second image regions 730 on the sample image 520 coincides with the position of the set of first image regions 720 on the first image 530. By replacing the set of first image regions 720 with a set of second image regions 730, embodiments of this disclosure can further improve the processing quality of the model.

[0075] In some embodiments, server 130 may also determine a set of third image regions 740 on a sample image 520 associated with the first image 530, and may further adjust the attribute information of a set of first image regions 720 based on the attribute information of the set of third image regions 740.

[0076] continue Figure 7 For example, a set of third image regions 740 is associated with a set of first image regions 720. As an example, the position of the set of third image regions 740 on the sample image 520 coincides with the position of the set of first image regions 720 on the first image 530. In some embodiments, attribute information of the image regions may indicate the characteristics of the image regions, such as color, size, etc.

[0077] In some embodiments, the server 130 may, for example, adjust the attribute information of a set of first image regions 720 to be consistent with the attribute information of a set of third image regions 740.

[0078] The following example illustrates the two adjustment methods. Multiple sample images 520 are portrait images, and the processing effect of the reference model 525 is a makeup effect x. The preset effective range 710 of the makeup effect x is the face portion of the image. The area in the first image 530 where the makeup effect occurs is the face portion, the arm portion, and the background of the first image. That is, the arm portion and the background of the first image 530 constitute a group of first image regions 720. After processing by the reference model 525, color a of the arm portion changes to color b, and background A changes to background B. As an example, the color can be attribute information of the area where the arm portion is located. After further adjustment, color b of the arm portion in the first image 530 changes to color a, and background B changes to background A. The change from color b to color a of the arm portion is achieved by adjusting the attribute information of the first image region 720 based on the attribute information of the third image region 740 in the sample image 520. The change from background B to background A is achieved by replacing the first image region 720 with the second image region 730.

[0079] Continue to refer to Figure 5 Server 130 can obtain multiple second images by adjusting multiple first images 530. Furthermore, server 130 can pair the multiple second images with corresponding sample images 520 to construct a second sample set 545.

[0080] In box 630, server 130 uses the second sample set 545 to train target model 560.

[0081] In some embodiments, similar to the training process of reference model 525, server 130 may use second sample set 545 to train second pre-trained model 550 to obtain target model 560.

[0082] As an example, the second pre-trained model 550 may include a machine learning model associated with the target effect, examples of which may include, but are not limited to, GANs (Generative Adversarial Networks). Taking makeup effects as an example, different makeup effects may be associated with different pre-trained GAN models.

[0083] In some embodiments, the processing effect achieved by the second pre-trained model 550 can be of the same type as the target effect. By training the second pre-trained model 550 using the second sample set 545, the time required to train the second pre-trained model 550 using the second sample set 545 can be saved. For example, if the target effect is a makeup effect A, the processing effect achieved by the second pre-trained model 550 can be another makeup effect B. As an example, the server 130 can select a model with a similar processing effect from among multiple models that achieve the same effect as the second pre-trained model 550. Embodiments of this disclosure can reduce the training cost of the model and improve the training efficiency of the model by selecting a second pre-trained model 550 with a related effect.

[0084] Additionally, when training the second pre-trained model 550 based on the second sample set 545, the server 130 can determine a training template 510 associated with the second pre-trained model 550 to further shorten the training time. In some instances, the training template 510 can be a combination of multiple hyperparameters. By using different training templates 510, embodiments of this disclosure effectively reduce the debugging process during model training and lower the training cost of the model.

[0085] In this way, server 130 can train a second pre-trained model 550 using a second sample set 545 to obtain a reference model 525.

[0086] Based on the process described above, embodiments of this disclosure train a reference model 525 using a first pre-trained model 515 associated with the target effect to construct a second sample set 545 for training the target model 560. These embodiments can shorten the time required to train the target model 560, thus improving the efficiency of training the target model 560. Furthermore, the quality of the samples is further improved by using a training template 540 matched with the first pre-trained model 515 to train the reference model 525, and by filtering and adjusting the first image 530, thereby enhancing the processing quality of the target model.

[0087] Example Devices and Apparatus

[0088] Embodiments of this disclosure also provide corresponding apparatus for implementing the above methods or processes. Figure 8A schematic structural block diagram of an example apparatus 800 for processing media content according to certain embodiments of the present disclosure is shown. Apparatus 800 may be implemented as or included in terminal device 110. Various modules / components in apparatus 800 may be implemented by hardware, software, firmware, or any combination thereof.

[0089] like Figure 8 As shown, the apparatus 800 includes: an acquisition module 810 configured to acquire first media content; a processing module 820 configured to apply a target effect to the first media content using a target model to generate second media content; and a providing module 830 configured to provide the second media content, wherein the target model is trained based on the following process: training a pre-trained model associated with the target effect using a first sample set to determine a reference model; constructing a second sample set based on the processing results of the reference model for multiple sample images; and training the target model using the second sample set.

[0090] In some embodiments, the multiple sample images include real images and synthetic images.

[0091] In some embodiments, constructing a second sample set based on the processing results of a reference model on multiple sample images includes: processing multiple sample images using a reference model to generate multiple first images corresponding to the multiple sample images; and constructing a second sample set based on the multiple first images.

[0092] In some embodiments, constructing a second sample set based on a plurality of first images includes: filtering at least one image from the plurality of first images that does not meet a preset condition to obtain a plurality of second images; and constructing a second sample set based on the plurality of second images and corresponding sample images.

[0093] In some embodiments, filtering at least one image that does not meet a preset condition from a plurality of first images includes: detecting a set of feature points in each first image that are associated with a preset object; and filtering at least one image that does not meet a preset condition from a plurality of first images based on a plurality of sets of feature points, wherein the preset condition is related to the number and / or positional relationship of a set of feature points.

[0094] In some embodiments, constructing a second sample set based on a plurality of first images includes: determining a set of first image regions in the plurality of first images that are independent of the target effect based on a preset effective range of the target effect; adjusting a set of first image regions in the plurality of first images based on the plurality of sample images; and constructing a second sample set based on the adjusted plurality of first images.

[0095] In some embodiments, adjusting a set of first image regions among a plurality of first images based on a plurality of sample images includes: determining a set of second image regions on a sample image associated with a first image, the set of second image regions being associated with a set of first image regions; and replacing a set of first image regions with the set of second image regions.

[0096] In some embodiments, adjusting a set of first image regions among a plurality of first images based on a plurality of sample images includes: determining a set of third image regions on a sample image associated with a first image, the set of third image regions being associated with a set of first image regions; and adjusting the attribute information of a set of first image regions based on attribute information of the set of third image regions.

[0097] In some embodiments, the pre-trained model is a first pre-trained model, and training the target model includes: using a second sample set to train a second pre-trained model associated with the target effect to obtain the target model.

[0098] like Figure 9 As shown, electronic device 900 is in the form of a general-purpose electronic device. Components of electronic device 900 may include, but are not limited to, one or more processors or processing units 910, memory 920, storage device 930, one or more communication units 940, one or more input devices 950, and one or more output devices 960. Processing unit 910 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 920. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of electronic device 900.

[0099] Electronic device 900 typically includes multiple computer storage media. Such media can be any accessible media that is accessible to electronic device 900, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 920 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 930 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media that can be used to store information and / or data and can be accessed within electronic device 900.

[0100] Electronic device 900 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 9As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 920 may include computer program product 925 having one or more program modules configured to perform various methods or actions of various embodiments of this disclosure.

[0101] The communication unit 940 enables communication with other electronic devices via a communication medium. Additionally, the functionality of the components of the electronic device 900 can be implemented using a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, the electronic device 900 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network node.

[0102] Input device 950 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 960 can be one or more output devices, such as a monitor, speaker, printer, etc. Electronic device 900 can also communicate with one or more external devices (not shown) via communication unit 940 as needed. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with electronic device 900, or with any device that enables electronic device 900 to communicate with one or more other electronic devices (e.g., network card, modem, etc.). Such communication can be performed via input / output (I / O) interface (not shown).

[0103] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. According to an exemplary implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above.

[0104] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0105] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0106] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0107] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0108] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method for processing media content, comprising: Obtain first-hand media content; The target effect is applied to the first media content using a target model to generate the second media content; as well as The second media content is provided, wherein the target model is trained based on the following process: training a pre-trained model associated with the target effect using a first sample set to determine a reference model; constructing a second sample set based on the processing results of the reference model for multiple sample images; and training the target model using the second sample set.

2. The method according to claim 1, wherein, The multiple sample images include real images and synthetic images.

3. The method according to claim 1, wherein, The construction of the second sample set based on the processing results of the reference model for multiple sample images includes: The reference model is used to process the plurality of sample images to generate a plurality of first images corresponding to the plurality of sample images; and The second sample set is constructed based on the plurality of first images.

4. The method according to claim 3, wherein, The construction of the second sample set based on the plurality of first images includes: Filter at least one image that does not meet the preset conditions from the plurality of first images to obtain a plurality of second images; and The second sample set is constructed based on the plurality of second images and the corresponding sample images.

5. The method according to claim 4, wherein, The step of filtering at least one image that does not meet the preset conditions from the plurality of first images includes: Detect a set of feature points in each of the first images that are associated with a preset object; and Based on a plurality of the set of feature points, at least one image that does not meet the preset conditions is filtered from the plurality of first images, wherein the preset conditions are related to the number and / or positional relationship of the set of feature points.

6. The method according to claim 3, wherein, The construction of the second sample set based on the plurality of first images includes: Based on the preset range of the target effect, a set of first image regions independent of the target effect is determined among the plurality of first images; Based on the plurality of sample images, adjust the set of first image regions in the plurality of first images; and The second sample set is constructed based on the adjusted plurality of first images.

7. The method according to claim 6, wherein, The step of adjusting the set of first image regions in the plurality of first images based on the plurality of sample images includes: Determine a set of second image regions on a sample image associated with the first image, the set of second image regions being associated with the set of first image regions; and The first set of image regions is replaced by the second set of image regions.

8. The method according to claim 6, wherein, Based on the plurality of sample images, adjusting the set of first image regions in the plurality of first images includes: Determine a set of third image regions on a sample image associated with the first image, the set of third image regions being associated with the set of first image regions; and Based on the attribute information of the third set of image regions, the attribute information of the first set of image regions is adjusted.

9. The method according to claim 1, wherein, The pre-trained model is a first pre-trained model, and training the target model includes: Using the second sample set, a second pre-trained model associated with the target effect is trained to obtain the target model.

10. An apparatus for processing media content, comprising: The acquisition module is configured to acquire the first media content; The processing module is configured to apply the target effect to the first media content using the target model to generate the second media content; as well as A providing module is configured to provide the second media content, wherein the target model is trained based on the following process: training a pre-trained model associated with the target effect using a first sample set to determine a reference model; constructing a second sample set based on the processing results of the reference model for multiple sample images; and training the target model using the second sample set.

11. An electronic device, comprising: At least one processing unit; as well as At least one memory, coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, which, when executed by the at least one processing unit, cause the electronic device to perform the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, the computer program being executable by a processor to implement the method according to any one of claims 1 to 9.

Images