Method and system for performing editing operations on media

Abstract

Methods and systems for performing editing operations on media are provided. A method includes receiving at least one reference medium and at least one target medium, identifying at least one primary editing attribute of the at least one reference medium, and performing a compatibility check to determine compatibility of the at least one target medium with the at least one primary editing attribute of the at least one reference medium. Based on the results of the compatibility check, at least one compatible editing attribute is selected from the at least one primary editing attribute, and the at least one compatible editing attribute is transferred from the at least one reference medium to the at least one target medium.

Description

Technical Field

[0001] The present disclosure relates generally to the field of processing media, and more particularly, to performing editing operations on media based on selection of compatible editing attributes. Background Art

[0002] Currently, there are several media editing applications that can be used to apply editing attributes to media that enhance the media's aesthetics (e.g., various effects / colors, artistic textures for photos, fun stickers, etc.). Summary of the invention

[0003] Technical issues

[0004] Existing media editing applications have their own editing attribute sets for selection. Therefore, when a user needs to apply similar editing attributes to new media, the user must download a specific media editing application and follow specific steps for applying similar editing attributes to the new media.

[0005] Furthermore, if multiple editing attributes (e.g., effects, background blur, contrast, etc.) are applied to a media to produce a pleasing effect, and the user wants to recreate the same effect on another media, the user must remember and perform the same steps on the desired media. This may result in the user having to perform complex editing operations. Furthermore, due to the complex editing operations, the user may not be able to create a similar effect on the desired media.

[0006] Furthermore, when a user wants to recreate a similar memory with existing content received via social media, the user may not be aware of a media editing application that can create such an effect on the desired media. Furthermore, an amateur or average user may not be technically aware of the complex editing operations to be performed on the media.

[0007] Furthermore, existing media editing applications enable users to perform only limited operations, such as transferring textures from one medium to another and viewing a list of editing operations performed on an image.

[0008] Solution to the problem

[0009] The present disclosure is directed to solving the above-mentioned problems and disadvantages, and to providing at least the advantages described below.

[0010] According to one aspect of the present disclosure, a method is provided, which includes receiving, by a receiving unit, at least one reference medium and at least one target medium, wherein the at least one reference medium includes at least one edited part; identifying, by a processing engine, at least one main editing attribute of the at least one reference medium; performing, by the processing engine, a compatibility check to determine the compatibility of at least one target medium with at least one main editing attribute of the at least one reference medium; selecting, by the processing engine, at least one compatible editing attribute from the at least one main editing attribute based on a result of the compatibility check; and transmitting, by an editing transmission engine, the selected at least one compatible editing attribute from the at least one reference medium to the at least one target medium.

[0011] According to another aspect of the present disclosure, an electronic device is provided, the electronic device including a receiving unit configured to receive at least one reference medium and at least one target medium, wherein the at least one reference medium includes at least one edited portion. The electronic device also includes a processing engine configured to identify at least one primary editing attribute of the at least one reference medium, perform a compatibility check to determine the compatibility of at least one target medium with at least one primary editing attribute of the at least one reference medium, and select at least one compatible editing attribute from the at least one primary editing attribute based on the result of the compatibility check. In addition, the electronic device also includes an editing transfer engine configured to transfer the selected at least one compatible editing attribute from the at least one reference medium to the at least one target medium.

[0012] Beneficial Effects

[0013] One aspect of the present disclosure is to provide a method and / or apparatus for editing target media by transferring compatible editing properties from reference media to the target media.

[0014] One aspect of the present disclosure is to provide a method and / or apparatus for selecting compatible editing properties of reference media for use in editing target media. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The above and other aspects, features and advantages of certain embodiments of the present disclosure will become apparent from the following description taken in conjunction with the accompanying drawings, in which:

[0016] Figure 1 is a system for performing editing operations on media according to an embodiment;

[0017] Figure 2A is a block diagram illustrating various elements of a processing engine for selecting compatible attributes of reference media to edit target media according to an embodiment;

[0018] Figure 2Bis a block diagram illustrating various elements of an edit delivery engine for applying compatible edit attributes to target media according to an embodiment;

[0019] Figure 3 is a flow chart illustrating a method for performing editing operations on media according to an embodiment;

[0020] Figure 4A is a flow chart illustrating a method for selecting a primary editing attribute of a reference media according to an embodiment;

[0021] Figure 4B is a flow chart illustrating a method for performing a compatibility check on a target media according to an embodiment;

[0022] Figure 5 is a block diagram showing various modules of an electronic device for performing an editing operation on a media according to an embodiment;

[0023] Figure 6 An edit classifier unit including a convolutional neural network (CNN) for determining edit attributes of reference media is shown according to an embodiment;

[0024] Figure 7 A processing engine and an edit delivery engine for selecting compatible edit attributes of reference media and applying the compatible edit attributes of the reference media to target media according to an embodiment are shown;

[0025] Fig. 8A A texture unit for transferring texture attributes from a reference medium to a target medium is shown according to an embodiment;

[0026] Figure 8B A template unit for transferring template attributes from a reference medium to a target medium according to an embodiment is shown;

[0027] Figure 8C An effects unit for transferring effect properties from a reference medium to a target medium is shown according to an embodiment;

[0028] Fig.9A is a diagram illustrating selecting one or more compatible editing properties of reference media to edit target media according to an embodiment;

[0029] Fig. 9B is a diagram illustrating selecting one or more compatible editing properties of reference media to edit target media according to an embodiment;

[0030] Fig. 9C is a diagram illustrating selecting one or more compatible editing properties of reference media to edit target media according to an embodiment;

[0031] Fig.10 is a flow chart illustrating the transfer of compatible editing attributes from reference media to target media according to an embodiment;

[0032] Fig.11A is a diagram illustrating selecting at least one primary editing attribute of a reference media according to an embodiment;

[0033] Fig. 11B is a diagram illustrating selecting at least one primary editing attribute of a reference media according to an embodiment;

[0034] Fig. 12A is a diagram showing a compatibility check process performed on a target media according to an embodiment;

[0035] Fig. 12B is a diagram showing a compatibility check process performed on a target media according to an embodiment;

[0036] Fig.13A is a diagram illustrating the transfer of compatible editing attributes from reference media to target media according to an embodiment;

[0037] Fig. 13B is a diagram illustrating the transfer of compatible editing attributes from reference media to target media according to an embodiment;

[0038] Fig. 13C is a diagram illustrating the transfer of compatible editing attributes from reference media to target media according to an embodiment;

[0039] Fig.14 is a diagram illustrating levels of controlling compatible editing attributes transferred from reference media to target media, according to an embodiment;

[0040] Fig.15A is a diagram illustrating the transfer of at least one (or more) compatible editing attributes from a reference medium to only a specific portion of a target medium, according to an embodiment;

[0041] Fig. 15B is a diagram illustrating the transfer of at least one (or more) compatible editing attributes from a reference medium to only a specific portion of a target medium, according to an embodiment;

[0042] Fig.16 is a diagram illustrating storing the resulting intermediate frames while transferring compatible editing attributes from reference media to target media, according to an embodiment;

[0043] Fig.17 is a diagram illustrating the transfer of compatible editing attributes from reference media present on a first electronic device to target media present on a second electronic device according to an embodiment;

[0044] Fig.18 is a diagram illustrating the transfer of compatible editing attributes from a reference image to a target video according to an embodiment;

[0045] Fig.19 is a diagram illustrating the transfer of compatible editing attributes from a reference image to a camera preview frame according to an embodiment;

[0046] Fig. 20 is a diagram illustrating selection of reference media from social media according to an embodiment;

[0047] Fig.21 is a diagram illustrating the transfer of facial enhancement attributes from reference media to target media according to an embodiment;

[0048] Fig. 22 is a diagram illustrating the transfer of compatible editing attributes from a plurality of reference media to a target media according to an embodiment; and

[0049] Fig.23 is a diagram illustrating receiving input of a user selection for transferring compatible editing attributes from reference media to target media, according to an embodiment. DETAILED DESCRIPTION

[0050] Various embodiments of the present disclosure are described with reference to the accompanying drawings. However, the various embodiments of the present disclosure are not limited to specific embodiments, and it should be understood that various modifications, equivalents and / or substitutions may be made to the embodiments described herein. With respect to the description of the accompanying drawings, similar components may be marked with similar reference numerals.

[0051] Embodiments herein disclose methods and systems for performing editing operations on media.

[0052] It is an object of an embodiment to edit target media by transferring compatible editing properties from reference media to the target media.

[0053] Another object of an embodiment is to select compatible editing properties of a reference medium for use in editing a target medium.

[0054] A method according to an embodiment includes receiving reference media and target media, wherein the reference media includes at least one medium and at least one edited portion of the medium. In addition, the method includes identifying one or more primary edited attributes of the reference media. In addition, the method includes performing a compatibility check to determine compatibility of the target media with one or more primary edited attributes of the one or more reference media. Based on the results of the compatibility check, a compatible edited attribute is selected from the one or more primary attributes of the reference media. In addition, the method includes transferring the compatible edited attributes from the reference media to the target media.

[0055] Figure 11 is a system 100 for performing editing operations on media according to an embodiment. The system includes an electronic device 102, a network 104, and an external source 106. The electronic device 102 may be, but is not limited to, a mobile phone, a smart phone, a tablet, a phablet, a personal digital assistant (PDA), a laptop, a computer, a wearable device, an Internet of Things (IoT) device, a vehicle infotainment system, a medical device, a camera, or any other device that supports media editing. Examples of media may be, but are not limited to, images, videos, graphic interchange formats (GIFs), animations, etc.

[0056] The electronic device 102 can communicate with an external source 106 via a network 104 for receiving media. Examples of networks may be, but are not limited to, the Internet, a wired network, and a wireless network (Wi-Fi network, cellular network, Wi-Fi hotspot, Bluetooth TM 、Zigbee TM and Near Field Communication (NFC). External source 106 may include media. Examples of external source 106 may be, but are not limited to, electronic devices, cameras, servers, databases, cloud storage, social media, and websites. Electronic device 102 includes receiving unit 108, processing engine 110, editing and transmitting engine 112, display unit 114, and storage unit 116.

[0057] The receiving unit 108 may be configured to receive at least two media from a user for performing an editing operation. The at least two media may exist in the electronic device 100 or may be obtained from one or more external sources 106. The at least two media include a reference media (the media from which the editing operation is copied) and a target media (the media to which the editing operation is applied). The reference media includes at least one portion that has been edited (previously edited or being edited in real time).

[0058] The receiving unit 108 may receive input from the user for initiating an editing operation on the target media. Examples of the input may include, but are not limited to, gestures (gaze, air gestures, touch, tap, click, press, swipe, hold, drag and drop), voice commands, gestures performed using a stylus (S-pen), device bending angles, and tapping two devices (NFC / Bluetooth).

[0059] The processing engine 110 may be configured to process the reference media to identify editing classes / attributes of the reference media. Editing attributes may be, but are not limited to, texture attributes, effect attributes, template attributes, and selfies. Examples of effect attributes may be pixel-level changes, such as, but not limited to, shatter effects, temperature changes, brightness, and contrast. Examples of template attributes may be, but are not limited to, static stickers, expressions, fun shots, and collages. Examples of selfies may be, but are not limited to, face beautification levels, slimming series, big eyes effects, and animated stickers.

[0060] The processing engine 110 may determine the primary editing attribute based on the editing attribute of the reference media. After determining the primary editing attribute, the processing engine 110 may perform a compatibility check on the target media to determine the compatibility of the target media with the primary editing attribute of the reference media. Based on the result of the compatibility check, the processing engine 110 may select a compatible editing attribute from the primary editing attributes based on the result of the compatibility check.

[0061] The editing and transferring engine 112 may be configured to edit the target media by applying / transferring the compatible editing attributes of the reference media to the target media. The display unit 114 may be configured to display the edited target media on a display screen of the electronic device 100 .

[0062] The storage unit 116 may be configured to store reference media, target media, master editing properties and compatible editing properties of the reference media, and edited target media. The storage unit 116 may be at least one of a file server, a data server, a server, a cloud, and a memory. The memory may include one or more computer-readable storage media and non-volatile storage elements, such as magnetic hard disks, optical disks, floppy disks, flash memory, electrically programmable memory (EPROM) or electrically erasable and programmable (EEPROM) memory. In addition, in some examples, the memory may be considered to be a non-transitory storage medium. The term "non-transitory" may mean that the storage medium is not implemented as a carrier or propagation signal. However, the term "non-transitory" should not be interpreted as meaning that the memory is non-removable. In some examples, the memory may be configured to store a larger amount of information than the memory. In some examples, the non-transitory storage medium may store data that may change over time (e.g., in a random access memory (RAM) or cache).

[0063] Figure 1 Components of system 100 are shown, but in other embodiments, system 100 may include additional components, or fewer components. In addition, the labels or names of the components are for illustration purposes only and do not limit the scope of the embodiments. One or more components can be combined together to perform the same or substantially similar functions in system 100.

[0064] 2A is a block diagram showing various units of the processing engine 110 for selecting compatible attributes of reference media to edit target media according to an embodiment. The processing engine 110 includes an edit classifier unit 202, an attribute filter unit 204, a compatibility check unit 206, and an attribute selection unit 208.

[0065] The edit classifier unit 202 may be configured to determine edit attributes of the reference media. The edit attributes may be classified into predefined attributes, such as, but not limited to, texture attributes, template attributes, effect attributes, and selfie attributes. The edit classifier unit 202 may also add all edits detected in the reference media, thereby expanding the attribute classification.

[0066] The edit classifier unit 202 may use feature extraction techniques, CNNs, or any other machine learning networks to identify the edit attributes of the reference media. The embodiments herein may be further explained with consideration of a CNN for identifying the edit attributes of the reference media, but it is apparent to one of ordinary skill in the art that any other form of feature extraction techniques may be used to identify the edit attributes. The CNN may be based on a feature extraction network, such as, but not limited to, a visual geometry group (VGG16) neural network and Inception. In addition, the CNN may include a deep learning algorithm.

[0067] Upon receiving the reference media from the receiving unit 202, the edit classifier unit 202 feeds the reference media to the CNN. The CNN may analyze at least one of the portion of the reference media and the portion of the reference media to identify the edit attribute of the reference media.

[0068] The edit classifier unit 202 may also be configured to calculate the probability of the edit attribute of the reference media and the probability of the edit attribute of at least one of the portions of the reference media. The edit classifier unit 202 may use a CNN to calculate the probability of the edit attribute. The edit classifier unit 202 may use a softmax function of a CNN, which may represent a probability distribution over N different possible results (probabilities of the edit attributes).

[0069] The attribute filter unit 204 may be configured to determine the primary editing attribute based on the editing attributes of the reference media. The attribute filter unit 204 analyzes the probabilities of the editing attributes of the reference media to determine the primary editing attributes of the reference media. For example, when the probability of the texture attribute and the probability of the template attribute are equal to or higher than the probability of the effect attribute, the attribute filter unit 204 determines that the texture attribute and the template attribute may be the primary editing attributes. In another example, when the probability of the texture attribute is higher than the probability of the template attribute and the probability of the effect attribute, the attribute filter unit 204 determines that the texture attribute may be the primary editing attribute. In yet another example, when the probability of the texture attribute, the probability of the template attribute, and the probability of the effect attribute are the same, the attribute filter unit 204 determines that the texture attribute, the template attribute, and the effect attribute may be the primary editing attributes.

[0070] The compatibility checking unit 206 may be configured to perform a compatibility check on the target media. The compatibility checking unit 206 may receive the target media from the receiving unit 108 and process at least one portion of the target media to identify a source attribute of the target media. The source attribute may be, but is not limited to, a texture, a template, an effect, and a selfie. The compatibility checking unit 206 may select at least one portion of the target media for processing based on metadata associated with the target media. The compatibility checking unit 206 may select the portion of the target media for processing upon receiving a selection input from the receiving unit 108. A selection input indicating a portion of the target image for processing may be received from a user.

[0071] The compatibility checking unit 206 compares the source attributes with the primary editing attributes of the reference media to determine compatibility of at least one of the target media or one of the portions of the target media with the primary editing attributes. In addition, based on the compatibility of the target media, the compatibility checking unit 206 may assign weight values ​​to the primary editing attributes. A higher weight value indicates that the target media is more compatible with the primary editing attributes of the reference media. A lower weight value indicates that the target media is less compatible with the primary editing attributes of the reference media.

[0072] Consider a scenario where a template attribute (i.e., a face sticker) and a texture attribute are primary editing attributes of a reference medium. The compatibility check unit 206 compares the source attribute of the target medium with the template attribute and the texture attribute of the reference medium. Based on the comparison result, the compatibility check unit 206 determines that the target medium has a primary texture and the target medium does not contain a template attribute (i.e., a face for applying a face sticker). Therefore, the compatibility check unit 206 assigns a lower weight value to the template attribute and a higher weight value to the texture attribute.

[0073] The attribute selection unit 208 may be configured to select a compatible editing attribute from the primary editing attributes of the reference media. The attribute selection unit 208 selects the compatible editing attribute based on the weight value assigned to the primary editing attribute. The compatible editing attribute may be a primary editing attribute with which at least one of the portions of the target media may be compatible. For example, a texture attribute and a template attribute may be determined as the primary editing attributes of the reference media. In addition, the texture attribute may have a higher weight value, and the template attribute may have a lower weight value. Therefore, the attribute selection unit 208 may select the texture attribute as the compatible editing attribute of the reference media.

[0074] Consider another example, where the effect attribute of the reference media is determined to be the primary editing attribute, and the target media has many dark pixels. Therefore, a lower weight value may be assigned to the effect attribute. In this case, the attribute selection unit 208 may not select a compatible editing attribute for editing the media as the target media because the effect attribute is determined to be incompatible with the effect attribute. Therefore, at least one relevant editing attribute may be identified to apply to the target media, rather than performing a blind copy (i.e., copying without a compatibility comparison) of the selected editing operation on the target media.

[0075] Figure 2A The components of the processing engine 110 are shown, but in other embodiments, the processing engine 110 may include additional components or fewer components. In addition, the labels or names of the units are only for illustrative purposes and do not limit the scope of the embodiments. One or more units can be combined together to perform the same or substantially similar functions in the processing engine 110.

[0076] Figure 2B 1 is a block diagram illustrating various elements of an edit transfer engine 112 for applying compatible edit attributes to target media, according to an embodiment. The edit transfer engine 112 transfers compatible edit attributes from reference media to at least one of a compatible portion of target media and a compatible target media. The edit transfer engine 112 may receive input (i.e., scribbles or irregular markings) from a user specifying a portion of the target media to which the compatible edit attributes may be applied. For example, the edit transfer engine 112 may apply at least one of the texture attribute, the template attribute, and the effect attribute to at least one of the portions of the target media in response to determining that at least one of the texture attribute, the template attribute, and the effect attribute may be a compatible edit attribute.

[0077] The editing delivery engine 112 may use suitable communication technologies (such as NFC, Bluetooth, Wi-Fi, Bluetooth TM Low Energy (BLE)) transfers compatible editing properties of reference media present on a first electronic device to target media present on a second electronic device.

[0078] The edit transfer engine 112 includes a texture unit 210, a template unit 212, and an effect unit 214. The texture unit 210 may be configured to apply texture properties of the reference media to at least one of a portion of the target media and the entire target media. The texture unit 210 uses a combination of an autoencoder neural network, a texture pattern calibrator, and an autodecoder to transfer the texture properties of the reference media to at least one of a portion of the target media and the entire target media.

[0079] The template unit 212 may be configured to apply the template attributes to at least one of a portion of the target media and the entire target media. The template unit 212 uses an image segmentation method to extract a mask (e.g., a face, an expression, a sticker, etc.) associated with the template attributes of the reference media. The template unit 212 matches the template attributes with a template stored in at least one of a local template database and a cloud template database. Based on a successful match, the template unit 212 transmits the template attributes to at least one of a portion of the target media and the entire target media.

[0080] The effects unit 214 may be configured to apply the effect attributes to at least one of a portion of the target media and the entire target media. The effects unit 214 may use color conversion algorithms and image filter effect algorithms to mimic and replicate pixel-level color changes from the reference media to the target media.

[0081] Figure 2B The components of the edit delivery engine 112 are shown, but it should be understood that the edit delivery engine 112 may include additional components or fewer components. In addition, the labels or names of the components are only for illustrative purposes and do not limit the scope of the embodiments. One or more components can be combined together to perform the same or substantially similar functions in the edit delivery engine 112.

[0082] Figure 3 is a flow chart 300 illustrating a method for performing editing operations on media, according to an embodiment.

[0083] At step 302, the method includes receiving, by a receiving unit 108 of an electronic device 102, reference media and target media. The reference media may include at least one of a portion of media and media that was previously edited. At step 304, the method includes identifying, by a processing engine 110 of the electronic device 102, a primary editing attribute of the reference media.

[0084] At step 306, the method includes performing a compatibility check, by the processing engine 110, to determine compatibility of the target media with the primary editing attributes. At step 308, the method includes selecting, by the processing engine 110, one or more compatible editing attributes from the primary editing attributes based on the results of the compatibility check. The processing engine 110 may select the compatible editing attributes based on weight values ​​assigned to the primary editing attributes. The weight values ​​may be assigned to the primary editing attributes based on the results of the compatibility check.

[0085] At step 310, the method includes transferring, by edit transfer engine 112, compatible edit attributes from the reference media to the target media. Edit transfer engine 112 may transfer the compatible edit attributes to a specific portion of the target media. The specific portion may be a portion of the target image that is compatible with the primary edit attributes of the reference media.

[0086] The various actions, actions, blocks or steps in the method and flowchart 300 may be performed in the order presented, in a different order or simultaneously. In addition, some of the actions, actions, blocks or steps may be omitted, added, modified or skipped without departing from the scope of the present disclosure.

[0087] Figure 4A is a flow chart 400a illustrating a method for selecting a primary editing attribute of a reference media, according to an embodiment.

[0088] At step 402, the method includes determining, by processing engine 110, an editing attribute of at least one of the reference media or at least one of the portions of the reference media. At step 404, the method includes calculating, by processing engine 110, a probability of the editing attribute of at least one of the reference media or at least one of the portions of the reference media. Processing engine 110 uses a CNN to determine the editing attributes and the probabilities of the editing attributes.

[0089] At step 406, the method includes analyzing the probabilities of the editing attributes to determine the primary editing attributes of the reference media by the processing engine 110. The processing engine 110 selects the editing attributes with higher probabilities as the primary editing attributes of the reference media.

[0090] The various actions, actions, blocks or steps in the method and flow chart 400a can be performed in the order presented, in a different order or simultaneously. In addition, some of the actions, actions, blocks or steps may be omitted, added, modified or skipped without departing from the scope of the present disclosure.

[0091] Figure 4B is a flow chart 400b illustrating a method for performing a compatibility check on a target media according to an embodiment.

[0092] At step 408, the method includes processing, by the processing engine 110, at least one portion of the target media to determine source attributes. At step 410, the method includes comparing, by the processing engine 110, the source attributes to the master editing attributes of the target media to determine compatibility of at least one of the portions of the target media with the master editing attributes.

[0093] At step 412, the method includes assigning, by the processing engine 110, a weight value to the primary editing attribute based on the compatibility of the portion of the target media and at least one of the target media with the primary editing attribute. When at least one of the portions of the target media is compatible with the primary editing attribute, the processing engine 110 assigns a higher weight value to the primary editing attribute. When at least one of the portions of the target media is incompatible with the primary editing attribute, the processing engine 110 assigns a lower weight value to the primary editing attribute.

[0094] Various actions, actions, blocks or steps in the method and flow chart 400b can be performed in the order presented, in a different order or simultaneously. In addition, some of the actions, actions, blocks or steps may be omitted, added, modified or skipped without departing from the scope of the present disclosure.

[0095] Figure 5 1 is a block diagram showing various modules of an electronic device 102 for performing an editing operation on a target media according to an embodiment. Embodiments of the present disclosure allow the electronic device 102 to edit the media by intelligently selecting compatible editing attributes of the reference media. The electronic device 102 may include a processing engine 110 for selecting compatible editing attributes of the reference media and an editing transmission engine 112 for transmitting the compatible editing attributes of the reference media to the target media.

[0096] like Figure 5 As shown, the edit classifier unit 202 of the processing engine 110 receives the reference media / pre-edited media and the target media. The edit classifier unit 202 determines the edit attributes (i.e., texture attributes, template attributes, and effect attributes) present in at least one of the reference media and the portion of the reference media. In addition, the edit classifier unit 202 calculates the probability of the edit attributes.

[0097] The attribute filter unit 204 of the processing engine 110 may determine the primary editing attribute from the editing attributes of the reference media based on the calculated probability of the editing attribute. The attribute filter unit 204 may also perform a compatibility check on the target media to determine the compatibility of the target media with the primary editing attribute. Based on the result of the compatibility check, the attribute filter unit 204 may determine the compatible editing attribute from the primary editing attribute.

[0098] The edit transfer engine 112 may transfer compatible edit properties from the reference media to the target media. Thus, only relevant edit properties may be applied to the target media.

[0099] Figure 6 An editing classifier unit 202 including a CNN is shown according to an embodiment for determining editing attributes of a reference media. The editing classifier unit 202 may use a CNN for determining editing attributes of a reference media (or a target media). The editing classifier unit 202 provides the reference media to the CNN. The CNN may include a feature learning mapping and classification part. The CNN may use a series of convolutional layers (Conv) and activation mapping layers (Relu) to learn the editing attributes / features of the reference media. In addition, the CNN may use a fully connected (FC) layer and a Sofmax classifier to classify the editing attributes into predefined attributes, such as texture attributes, template attributes, and effect attributes. In addition, the CNN may calculate the probability of the editing attributes of the reference media.

[0100] Figure 7 A processing engine 110 and an edit delivery engine 112 are shown for selecting compatible edit attributes of reference media and applying the compatible edit attributes of the reference media to target media according to an embodiment. The edit classifier unit 202 of the processing engine 110 provides information about the probabilities of the edit attributes of the reference media and the target media to the attribute filter unit 204.

[0101] Upon receiving the information about the probability of the editing attribute, the attribute filter unit 204 may determine the main editing attribute. The main editing attribute may be at least one of a texture attribute, a template attribute, and an effect attribute. After determining the main editing attribute, the compatibility check unit 206 may check the compatibility of the target media with respect to the main editing attribute. Based on the result of the compatibility check, the attribute selection unit 208 may select a compatible editing attribute from the main editing attributes. The compatible editing attribute may be at least one of a texture attribute, a template attribute, and an effect attribute.

[0102] For example, if the compatible editing attribute is determined to be a texture attribute, the texture unit 210 applies the texture attribute to the target media. If the compatible editing attribute is determined to be a template attribute, the template unit 212 applies the template attribute to the target media. Similarly, if the compatible attribute is determined to be an effect attribute, the effect unit 214 may apply the effect attribute to the target media.

[0103] Fig. 8A A texture unit 210 for transferring texture properties from a reference medium to a target medium is shown according to an embodiment. The texture unit 210 may use an autoencoder neural network, a texture pattern calibrator, and an autodecoder to apply the texture properties of the reference medium to the target medium. The autoencoder neural network and the autodecoder may include Conv and Relu.

[0104] Figure 8B A template unit 212 for transferring template attributes from a reference medium to a target medium is shown according to an embodiment. The template unit 212 may use Conv and Relu to extract a template mask for the reference medium. In addition, the template unit 212 may use a pattern matching method to match the extracted template mask with a template database. Based on a successful match, the template unit 212 may apply the extracted template mask to the target medium.

[0105] Figure 8C An effects unit 214 is shown for transferring effect attributes from reference media to target media according to an embodiment. The effects unit 214 can use color conversion algorithms to perform actions such as mimicking and replicating pixel-level color changes from the reference media to the target media. The effects unit can also apply image filtering effect algorithms to apply the effect attributes to the target media.

[0106] Fig.9A , Fig. 9B and Fig. 9C is a diagram illustrating selecting compatible editing properties of reference media for editing target media according to various embodiments.

[0107] Fig.9A is a diagram showing the selection of a texture attribute as a compatible editing attribute for editing a target medium. Fig.9A As shown, the editing classifier unit 202 of the processing engine 110 receives the reference media and identifies the editing attributes of the reference media. The editing attributes may be a texture attribute, a template attribute, and an effect attribute. In addition, the editing classifier unit 202 calculates the probabilities of the editing attributes of the reference media. For example, the probability of the texture attribute may be 0.85, the probability of the template attribute may be 0.01, and the probability of the effect attribute may be 0.05. Because the probability of the texture attribute is higher than the probability of the template attribute and the probability of the effect attribute, the attribute filter unit 204 of the processing engine 110 selects the texture attribute as the main editing attribute.

[0108] In addition, the compatibility checking unit 206 of the processing engine 110 may check the compatibility of the target media with the texture attribute. The compatibility checking unit 206 may determine that the target media is compatible with the texture attribute unit. Therefore, the compatibility checking unit 206 may assign a higher weight value to the texture attribute.

[0109] The attribute selection unit 208 of the processing engine 110 may select the texture attribute as the compatible edit attribute because the texture attribute has a higher weight value. Therefore, the texture unit 210 of the edit transfer engine 112 may transfer the texture attribute from the reference media to the target media.

[0110] Fig. 9B 2 is a diagram showing selection of effect attributes and template attributes as compatible editing attributes for editing target media. Fig. 9B As shown, the editing classifier unit 202 of the processing engine 110 receives the reference media and identifies the editing attributes of the reference media. The editing attributes may be a texture attribute, a template attribute, and an effect attribute. In addition, the editing classifier unit 202 calculates the probabilities of the editing attributes of the reference media. For example, the probability of the texture attribute may be 0.1, the probability of the template attribute may be 0.4, and the probability of the effect attribute may be 0.4. The attribute filter unit 204 selects the template attribute and the effect attribute as the main editing attributes because the probability of the template attribute and the probability of the effect attribute are equal to or higher than the probability of the texture attribute.

[0111] In addition, the compatibility checking unit 206 of the processing engine 110 may check the compatibility of the target media with the template attributes and the effect attributes. The compatibility checking unit 206 may determine that the target media is compatible with the template attributes and the effect attributes. Therefore, the compatibility checking unit 206 may assign a higher weight value to the template attributes and the effect attributes.

[0112] The attribute selection unit 208 of the processing engine 110 may select the template attribute and the effect attribute as compatible editing attributes based on the weight values. Thus, the template unit 212 and the effect unit 214 may transfer the template attribute and the effect attribute from the reference media to the target media, respectively.

[0113] Fig. 9C 2 is a diagram showing selection of an effect attribute as a compatible editing attribute for editing a target medium. Fig. 9C As shown, the edit classifier unit 202 receives reference media and identifies edit attributes of the reference media. The edit attributes may be texture attributes, template attributes, and effect attributes. In addition, the edit classifier unit 202 calculates the probabilities of the edit attributes of the reference media. For example, the probability of the texture attribute may be 0.1, the probability of the template attribute may be 0.4, and the probability of the effect attribute may be 0.4. The attribute filter unit 204 selects the template attribute and the effect attribute as the main edit attributes because the probability of the template attribute and the probability of the effect attribute are equal to or higher than the probability of the texture attribute.

[0114] In addition, the compatibility checking unit 206 may check the compatibility of the target media with the template attributes and the effect attributes. The compatibility checking unit 206 may determine that the target media is compatible with the effect attributes and incompatible with the template attributes. Therefore, the compatibility checking unit 206 may assign a higher weight value to the effect attributes and a lower weight value to the template attributes.

[0115] The attribute selection unit 208 may select the effect attribute as a compatible editing attribute based on the higher weight value. Therefore, the effect unit 214 may transfer the effect attribute from the reference media to the target media.

[0116] Fig.10 FIG. 4 is a flow chart illustrating the transfer of compatible editing attributes from reference media to target media according to an embodiment. Fig.10 As shown, at step 1001, the processing engine 110 selects a reference medium. At step 1002, the processing engine 110 selects a target medium. At step 1003, the processing engine 110 identifies the primary editing attribute of the reference medium. At step 1004, the processing engine 110 checks whether the primary editing attribute is clearly identifiable. In response to determining that the primary editing attribute is not clearly identifiable (1004: No), the processing engine 110 stops performing the editing operation on the target medium and does not proceed to step 1005.

[0117] In response to determining that the primary editing attributes are clearly identifiable (1004: YES), the processing engine 110 processes the target media and identifies the source attributes at step 1005. The processing engine 110 may also compare the source attributes with the primary editing attributes; assign weight values ​​to the primary editing attributes based on compatibility of the target media with the primary editing attributes; and select the primary editing attributes as compatible editing attributes based on the weight values ​​assigned to the primary editing attributes.

[0118] After selecting the compatible editing attributes, the edit transfer engine 112 converts the compatible editing attributes from the reference media to the target media in step 1006. In step 1007, the edited target media is displayed as a frame output on a display screen of the electronic device.

[0119] Fig.11A and Fig. 11B is a diagram illustrating selecting a primary editor attribute of a reference media according to various embodiments. The processing engine 110 may use a CNN to identify the primary editor attribute of the reference media. Fig.11A As shown, the processing engine 110 identifies the effect attribute (color effect) as the primary editing attribute in the reference medium. Therefore, only the color information can be transferred to the target medium.

[0120] In another example, if Fig. 11B As shown, the processing engine 110 identifies the effect attribute (ie, color effect) and the template attribute as the main editing attributes in the reference media. Therefore, the effect attribute and the template attribute can be applied to the target media.

[0121] Fig. 12A and Fig. 12B 1 is a diagram illustrating a compatibility check process performed on a target media according to an embodiment. The processing engine 110 may perform a compatibility check on the target media to determine the compatibility of the target media with the primary editing attributes of the reference media. Fig. 12A As shown, processing engine 110 identifies the effect attribute (i.e., adding bunny ears and whiskers to a face) and the template attribute from the reference media as primary editing attributes. In addition, processing engine 110 performs a compatibility check on the target media to determine compatibility with the effect attribute and the template attribute. Based on the compatibility check, processing engine 110 determines that the target media is compatible with the effect attribute and that the target media is not compatible with the template attribute. Therefore, processing engine 110 selects the effect attribute as a compatible editing attribute that can be applied to the target media.

[0122] like Fig. 12BAs shown, processing engine 110 identifies the texture attribute as a primary editing attribute of the reference media. Processing engine 110 checks the compatibility of the target media with the texture attribute. The target media may not be compatible with the texture attribute because the target media already has the texture attribute. Processing engine 110 does not select a compatible editing attribute for application on the target media. Therefore, the target media may be displayed without applying any editing attributes.

[0123] Fig.13A , Fig. 13B and Fig. 13C is a diagram illustrating the transfer of compatible editing attributes from reference media to target media, according to various embodiments.

[0124] Fig.13A is a diagram illustrating the transfer of texture attributes from a reference image to a target image. The processing engine 110 identifies the artistic texture (ie, texture attribute) of the reference image as a compatible edit attribute. The edit transfer engine 112 edits the target image by applying the artistic texture of the reference image to the target image.

[0125] Fig. 13B is a diagram showing the transfer of effect attributes from a reference image to a target image. The processing engine 110 identifies the effect attribute (color effect) of the reference image as a compatible edit attribute. The edit transfer engine 112 edits the target image by applying the effect attribute (ie, color effect) of the reference image to the target image.

[0126] Fig. 13C is a diagram showing the transfer of template attributes from a reference image to a target image. The processing engine 110 identifies the template attributes (i.e., stickers) of the reference image as compatible edit attributes. The edit transfer engine 112 edits the target image by applying the template attributes (i.e., stickers) of the reference image to the target image.

[0127] Fig.14 is a diagram showing control of the level of compatible editing attributes transferred from a reference medium to a target medium according to an embodiment. The edit transfer engine 112 may copy the compatible editing attributes from the reference image to the target image. Once the compatible editing attributes have been copied to the target image, the display unit 114 may display a slider on the display screen of the electronic device. The user may use the slider to change the level of the compatible editing attributes applied to the target image.

[0128] Fig.15A and Fig. 15B is a diagram illustrating transferring compatible edit attributes from reference media to only specific portions of target media according to various embodiments. The edit transfer engine 112 may apply compatible edit attributes to specific regions of interest (ROIs) / portions in a target image, or to the entire image.

[0129] The edit transfer engine 112 may receive input (touch, mark, or scribble) from a user designating a specific ROI (alternatively, the entire target image may be designated as the ROI) in the target image for performing an edit operation. Based on the input received from the user, the edit transfer engine 112 may apply compatible edit attributes to at least one of the specific ROIs in the target image, such as the foreground of the target image and the background of the image, such as Fig.15A shown.

[0130] In addition, the edit transfer engine 112 may apply compatible edit attributes of the reference image to specific compatible ROIs in the target image, such as Fig. 15B shown.

[0131] Fig.16 is a diagram illustrating storing resulting intermediate frames while transferring compatible editing attributes from reference media to target media, according to an embodiment.

[0132] If one or more edit attributes (e.g., texture or color) are identified, the processing engine 110 may enable the user to select to save and / or render each intermediate frame as a Graphics Interchange Format (GIF) file or video file, where each frame may capture each edit result. The final frame may have a Fig.16 The combined effect is shown. The user can discard or save the edited target media.

[0133] Fig.17 1 is a diagram illustrating the transfer of compatible editing attributes from reference media present on a first electronic device to target media present on a second device, according to an embodiment. For example, the reference media is present on the first electronic device, and the target media is present on the second electronic device. The edit transfer engine 112 of the first electronic device uses a communication mechanism such as NFC to transfer the edit operation (e.g., compatible editing attributes) to the target media present on the second electronic device.

[0134] Fig.18 is a diagram illustrating the transfer of compatible editing properties from a reference image to a target video, according to an embodiment. The compatible editing properties may be transferred from the reference image to the target video. The processing engine 110 may convert the video sequence to match the compatible editing properties / style of the reference image. The processing engine 110 may receive user input indicating the editing properties of the reference image that need to be transferred to the target video.

[0135] Fig.19 is a diagram illustrating the transfer of compatible editing attributes from a reference image to a camera preview frame according to an embodiment. Fig.19As shown, a user may select an image (e.g., media) having a unique style from a favorite database provided by the storage unit 116, where the user has added some images as favorite images in the storage unit 116. In addition, the user may select a style (e.g., compatible editing attributes) to be applied to a preview frame in the camera. The editing transfer engine 112 may transfer the style selected by the user to the preview frame. After transferring the style to the preview frame, a preview / target image may be captured by the camera. The target image may include the style selected by the user.

[0136] Fig. 20 1 is a diagram illustrating selection of reference media from social media, according to an embodiment. A user may download a plurality of images from an external source 106, such as various social media platforms. The plurality of images may be stored in a storage unit 116. In addition, the user may select a reference image and a target image from the plurality of images for applying the editing properties of the reference image to the target image. The processing engine 110 may determine a texture property of the reference image as a compatible editing property. After selecting the compatible editing property, the editing transfer engine 112 may transfer the texture property (e.g., the compatible editing property) from the reference image to the target image.

[0137] Fig.21 1 is a diagram illustrating the transfer of facial enhancement attributes from a reference medium to a target medium according to an embodiment. The edit transfer engine 112 may transfer the selfie attributes from the reference image to the target image. For example, the user may select the enhancements applied to the face present in the reference image as compatible edit attributes. Based on the user selection, the edit transfer engine 112 may apply the enhancements to the face present in the target image, such as Fig.21 shown.

[0138] Fig. 22 is a diagram showing the transfer of compatible editing attributes from multiple reference media to target media according to an embodiment. The electronic device may apply different editing attributes to different parts of a target image. For example, a user may select a first reference image and a second reference image. In addition, the user may select an editing attribute / style (i.e., a texture attribute) from the first reference media as a first editing attribute. The user may want to apply the first editing attribute to the background of the target image. In addition, the user may select an editing attribute (i.e., an effect attribute) from the second reference image as a second editing attribute. The user may select the second editing attribute to apply to the foreground of the target image.

[0139] Based on the user selection, the edit delivery engine 112 may apply a first edit attribute (i.e., a texture attribute) to the background of the target image and a second edit attribute (i.e., an effect attribute) to the foreground of the target image, such as Fig. 22 Thus, different editing attributes / styles can be applied to the foreground and background of the target image.

[0140] Fig.23 is a diagram illustrating receiving user selected input for transferring compatible editing attributes from reference media to target media, according to an embodiment. Fig.23 As shown, a user can capture an image and select the captured image as a reference image. In addition, the user can select one or more colors (i.e., effect attributes) of the reference image as editing attributes, which can be applied to subsequent images when capturing subsequent images. The edit transfer engine 112 can transfer the colors of the reference image (i.e., the previous image) to the image when capturing. Therefore, the captured image can include the color effects from the previously captured image.

[0141] Different camera options / modes of existing camera features may be combined into a mode / option that the user prefers.

[0142] Simple steps may be provided to combine different media options such as face beautification, effects, stickers, and collages into a one-step capture option using reference media.

[0143] Steps and editing time can be reduced to a single step to transfer the edit operation from the reference media to the target media.

[0144] A deep learning algorithm may be utilized to select editing properties of a reference media and apply the selected editing properties to the target media.

[0145] There may be no need to install multiple applications to apply edits.

[0146] The embodiments of the present disclosure may be implemented by at least one software program running on at least one hardware device and performing a network management function to control the elements. Figure 1 The elements shown in FIG. 2 may be at least one of a hardware device or a combination of a hardware device and a software module.

[0147] The present disclosure describes methods and systems for performing editing operations on media. Therefore, it should be understood that the scope of protection is extended to such a program, and in addition to a computer-readable device having a message therein, such a computer-readable storage device may contain program code means for implementing one or more steps of the method when the program is run on a server or a mobile device or any suitable programmable device.

[0148] The method may be implemented by a software program written in a very high speed integrated circuit hardware description language (VHDL) or another programming language, or implemented together with the software program. The software program may be implemented by one or more devices running VHDL or multiple software modules running on at least one hardware device. The device may be any type of portable device that can be programmed. The device may also include a hardware device, such as an application specific integrated circuit (ASIC), or a combination of hardware and software, such as an ASIC and a field programmable gate array (FPGA), or at least one microprocessor and at least one memory having a software module therein. The method described herein may be implemented partially in hardware and partially in software. Alternatively, the method may be implemented on different hardware devices using multiple central processing units (CPUs).

[0149] While the present disclosure has been particularly shown and described with reference to certain embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims and their equivalents.

Claims

1. A method for performing an editing operation on a media, the method comprising: receiving, by a receiving unit, at least one reference medium and at least one target medium, wherein the at least one reference medium includes at least one edited portion; identifying, by a processing engine, at least one primary editorial attribute of the at least one reference media; performing, by the processing engine, a compatibility check to determine compatibility of the at least one target media with the at least one primary editing attribute; selecting, by the processing engine, at least one compatible editing attribute from the at least one primary editing attribute based on a result of the compatibility check; and transferring, by an edit transfer engine, the selected at least one compatible edit attribute from the at least one reference medium to the at least one target medium, The performing of the compatibility check includes: processing at least one portion of the at least one target media to determine at least one source attribute; comparing the at least one source property to the at least one master editing property to determine compatibility of the at least one portion of the at least one target media with the at least one master editing property; and Based on the compatibility of the at least one portion of the at least one target media with the at least one primary editing property, a weight value is assigned to the at least one primary editing property, wherein the weight value is used to select the at least one compatible editing property of the at least one reference media.

2. The method according to claim 1, wherein: Identifying the at least one primary editing attribute comprises: determining at least one editorial property of at least one portion of the at least one reference media; calculating a probability of the at least one editing attribute of the at least one portion of the at least one reference media; and The at least one primary editorial attribute of the at least one reference media is identified by analyzing the probability of the at least one editorial attribute of the at least one portion of the at least one reference media.

3. The method according to claim 2, wherein: A convolutional neural network (CNN) is used to identify the at least one primary attribute.

4. The method of claim 1, further comprising analyzing at least one piece of metadata associated with the at least one target media to select the at least one portion of the at least one target media for processing.

5. The method of claim 1, further comprising receiving at least one input from at least one user to select the at least one portion of the at least one target media for processing.

6. An electronic device for performing an editing operation on a media, the electronic device comprising: a receiving unit configured to receive at least one reference medium and at least one target medium, wherein the at least one reference medium includes at least one edited portion; Processing engine, configured as: identifying at least one primary editorial attribute of the at least one reference media; performing a compatibility check to determine compatibility of the at least one target media with the at least one primary editing attribute; and selecting at least one compatible editing property from the at least one primary editing property based on a result of the compatibility check; and an edit transfer engine configured to transfer the selected at least one compatible edit attribute from the at least one reference medium to the at least one target medium, Wherein, when performing the compatibility check, the processing engine is configured as follows: processing at least one portion of the at least one target media to determine at least one source attribute; comparing the at least one source property to the at least one master editing property to determine compatibility of the at least one portion of the at least one target media with the at least one master editing property; and Based on the compatibility of the at least one portion of the at least one target media with the at least one primary editing property, a weight value is assigned to the at least one primary editing property, wherein the weight value is used to select the at least one compatible editing property of the at least one reference media.

7. The electronic device according to claim 6, wherein: The processing engine is further configured to: determining at least one editing property of at least one of at least one portion of the at least one reference medium and the at least one reference medium; calculating a probability of the at least one editing attribute of at least one of the at least one portion of the at least one reference medium and the at least one reference medium; as well as The at least one primary editorial attribute of the at least one reference medium is identified by analyzing the probability of the at least one portion of the at least one reference medium and the at least one editorial attribute of at least one of the at least one reference medium.

8. The electronic device according to claim 7, wherein: A convolutional neural network (CNN) is used to identify the at least one primary attribute.

9. The electronic device according to claim 6, wherein: The processing engine is further configured to analyze at least one piece of metadata associated with the at least one target media to select the at least one portion of the at least one target media for processing.

10. The electronic device according to claim 6, wherein: The processing engine is further configured to receive at least one input from at least one user to select the at least one portion of the at least one target media for processing.

Images