Multimedia editing method and device

Abstract

The embodiment of the invention provides a multimedia editing method, and the method comprises the steps: placing a basic material in a first track under the condition that the imported basic material is received, and the basic material comprises at least one of a picture, an audio and a video; obtaining a first feature of the basic material; obtaining a second feature of the decoration material; matching a decoration material with the basic material based on the first feature and the second feature, determining the decoration material matched with the basic material as a target decoration material, and determining a target position of the target decoration material placed on a second track, the second track being synchronized with the first track in time; and putting the target decoration material into the target position. According to the technical scheme, the effect of intelligent adsorption of the decoration materials can be achieved, and the multimedia editing efficiency is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of multimedia technology, and in particular to a multimedia editing method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] In existing multimedia editors, users usually need to manually drag decorative materials (such as stickers, audio, special effects, etc.) to the material track to match them with the user materials on the video track to achieve multimedia editing.

[0003] However, for each decorative material, the user needs to determine its position on the material track one by one and then manually drag it in, which is relatively inefficient.

[0004] It should be noted that the above content is not necessarily prior art, nor is it intended to limit the scope of patent protection of this application. Summary of the Invention

[0005] The embodiments of the present application provide a multimedia editing method, apparatus, computer device, computer-readable storage medium, and computer program product to solve or alleviate one or more of the technical problems raised above.

[0006] One aspect of an embodiment of the present application provides a multimedia editing method, the method comprising: Upon receiving the imported basic material, placing the basic material into the first track, the basic material comprising at least one of a picture, audio, and video; Obtaining a first feature of the basic material; Obtaining the second characteristic of the decoration material; matching the decorative material with the base material based on the first feature and the second feature, determining the decorative material that matches the base material as a target decorative material, and determining a target position for placing the target decorative material on a second track, where the second track is synchronized with the first track in time; The target decoration material is placed in the target position.

[0007] Optionally, obtaining the first feature of the basic material includes: A first feature of the target content of the basic material within a specific time range is obtained.

[0008] Optionally, the target content includes frame images and audio; Correspondingly, the step of obtaining the first feature of the target content of the basic material within a specific time range includes: Determining content features of the frame image using computer vision technology; Determining visual features of the frame image using image processing technology; Determining audio characteristics of the audio using audio processing technology; The first feature is obtained based on the content feature, the visual feature, and the audio feature.

[0009] Optionally, obtaining the second feature of the decoration material includes: The second feature of the decorative material is obtained through an artificial intelligence algorithm, where the second feature includes at least one of a content feature, a color feature, a shape feature, a dynamic feature, and a scene feature.

[0010] Optionally, the method further includes: Obtaining a matching analysis result of the target decoration material; When the target decoration material is placed in the target position, the matching analysis result is visually displayed on an editing interface.

[0011] Optionally, the method further includes: When an adjustment instruction for the target decoration material is received, the position of the target decoration material on the second track is adjusted according to the adjustment instruction.

[0012] Another aspect of an embodiment of the present application provides a multimedia editing device, the device comprising: A first inserting module is configured to insert the imported basic material into the first track upon receiving the imported basic material, wherein the basic material includes at least one of a picture, audio, and video; A first acquisition module, configured to acquire a first feature of the basic material; A second acquisition module, used to acquire a second feature of the decoration material; a determination module, configured to match a decorative material with the base material based on the first feature and the second feature, determine the decorative material that matches the base material as a target decorative material, and determine a target position for placing the target decorative material on a second track, wherein the second track is synchronized with the first track in time; The second placing module is used to place the target decoration material into the target position.

[0013] Another aspect of an embodiment of the present application provides a computer device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein: the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method described above.

[0014] Another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method described above is implemented.

[0015] Another aspect of an embodiment of the present application provides a computer program product, including a computer program, which implements the above-mentioned method when executed by a processor.

[0016] The above technical solution adopted in the embodiments of the present application may have the following advantages: By placing the imported basic material into the first track upon receiving it, obtaining the first feature of the basic material, obtaining the second feature of the decorative material, matching the decorative material with the basic material based on the first feature and the second feature, determining the decorative material that matches the basic material as the target decorative material, and determining the target position for placing the target decorative material in the second track, and placing the target decorative material into the target position, the target decorative material that matches the basic material can be determined based on the matching of the features, and the target decorative material can be automatically placed in the corresponding position, thereby achieving the effect of intelligent adsorption of decorative materials and effectively improving the efficiency of multimedia editing. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings illustrate exemplary embodiments and constitute a part of the specification. Together with the description of the specification, they serve to explain exemplary implementation of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of the claims. Throughout the drawings, the same reference numerals designate similar, but not necessarily identical, elements.

[0018] Figure 1 The following schematically shows an operating environment diagram of the multimedia editing method according to the first embodiment of the present application; Figure 2 The flowchart of the multimedia editing method according to the first embodiment of the present application is schematically shown; Figure 3 Schematically shows Figure 2 Flowchart of sub-steps of step S202; Figure 4 The following schematically illustrates a newly added process of the multimedia editing method according to the first embodiment of the present application; Figure 5 A block diagram schematically shows a multimedia editing device according to the second embodiment of the present application; and Figure 6 The following schematically shows a hardware architecture diagram of a computer device according to the third embodiment of the present application. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solutions and advantages of this application more clear, the present application is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0020] It should be noted that the descriptions of "first", "second", etc. in the embodiments of the present application are for descriptive purposes only and should not be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of such features. In addition, the technical solutions between the various embodiments can be combined with each other, but this must be based on the fact that they can be implemented by ordinary technicians in this field. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0021] In the description of this application, it should be understood that the numerical labels before the steps do not indicate the order in which the steps are executed. They are only used to facilitate the description of this application and to distinguish each step. Therefore, they cannot be understood as limitations on this application.

[0022] First, an explanation of the terms involved in this application is provided: Computer vision technology, a key branch of artificial intelligence, aims to enable computers to understand and interpret information in visual images or videos, simulating the functions of the human visual system. It uses algorithms and models to process, analyze, and understand digital images or videos, ultimately achieving tasks such as object detection, image classification, semantic segmentation, and pose estimation.

[0023] Image processing technology: It is a technology that analyzes, modifies and optimizes digital images through computer algorithms, aiming to improve image quality, extract useful information or achieve specific visual effects.

[0024] Audio processing technology: It is a technology that analyzes, modifies, enhances or converts audio signals through computer algorithms, aiming to improve audio quality, extract useful information or achieve specific functions.

[0025] Secondly, to facilitate those skilled in the art to understand the technical solutions provided in the embodiments of the present application, the relevant technologies are described below: In current multimedia editors, users typically need to manually determine the location of each decorative material on the material track, then manually drag it onto the multimedia editor's material track to align it with the user's material on the multimedia editor's video track. However, this method requires users to manually manipulate each decorative material, consuming a significant amount of time and effort, resulting in low efficiency.

[0026] To this end, embodiments of the present application provide a multimedia editing technology solution. This technology solution achieves intelligent adsorption of decorative materials by acquiring the characteristics of both user materials and decorative materials, determining decorative materials that match the user materials based on these characteristics, and automatically placing the decorative materials in the corresponding positions, thereby improving multimedia editing efficiency. See below for details.

[0027] Finally, for ease of understanding, an exemplary operating environment is provided below.

[0028] like Figure 1 As shown, the environment diagram includes a service platform 2, a network 4, and a client 6, wherein: The service platform 2 can be comprised of a single or multiple computing devices. These multiple computing devices can include virtualized computing instances. Virtualized computing instances can include virtual machines, such as emulations of computer systems, operating systems, servers, and the like. A computing device can load a virtual machine based on a virtual image and / or other data defining the specific software (e.g., operating system, specialized application, server) used for the emulation. As demand for different types of processing services changes, different virtual machines can be loaded and / or terminated on one or more computing devices. A hypervisor can be implemented to manage the use of different virtual machines on the same computing device.

[0029] The service platform 2 can be configured to communicate with clients 6 and the like via a network 4. The network 4 includes various network devices, such as routers, switches, multiplexers, hubs, modems, bridges, repeaters, firewalls, proxy devices, and / or the like. The network 4 can include physical links, such as coaxial cable links, twisted pair cable links, fiber optic links, combinations thereof, or wireless links, such as cellular links, satellite links, Wi-Fi links, and the like.

[0030] The service platform 2 can provide storage, reading, writing, querying, deleting and other services, such as providing multimedia editing services for clients.

[0031] Client 6 can be an electronic device running an operating system such as Windows, Android™, or iOS, such as a smartphone, tablet, laptop, virtual reality device, gaming device, set-top box, in-vehicle terminal, or smart TV. Based on these operating systems, various applications can be run, such as multimedia editing applications.

[0032] The client 6 may provide / configure a user access page for manipulating the service platform 2 or uploading objects, etc.

[0033] It should be noted that the above devices are exemplary, and the number and type of devices can be adjusted in different scenarios or according to different needs.

[0034] The technical solutions of the present application are described below through a number of embodiments. It should be noted that these embodiments can be implemented in a variety of different forms and should not be construed as being limited to the embodiments described herein.

[0035] Example 1 Figure 2 The flowchart of the multimedia editing method according to the first embodiment of the present application is schematically shown. It should be noted that the execution subject of the multimedia editing method of the embodiment of the present application can be a service platform or a client, and the following exemplary description is given using the client as an example.

[0036] like Figure 2 As shown, the multimedia editing method may include steps S200 to S208, wherein: Step S200: When the imported basic material is received, the basic material is placed into the first track, where the basic material includes at least one of a picture, audio, and video.

[0037] Step S202: Obtain the first feature of the basic material.

[0038] Step S204: obtaining the second feature of the decoration material.

[0039] Step S206, matching the decorative material with the basic material based on the first feature and the second feature, determining the decorative material that matches the basic material as the target decorative material, and determining the target position of the target decorative material on the second track, and the second track is synchronized with the first track in time.

[0040] Step S208: placing the target decoration material into the target position.

[0041] The multimedia editing method provided in this embodiment places the basic material into the first track upon receiving the imported basic material, obtains the first feature of the basic material, obtains the second feature of the decorative material, matches the decorative material with the basic material based on the first feature and the second feature, determines the decorative material that matches the basic material as the target decorative material, determines the target position for placing the target decorative material in the second track, places the target decorative material in the target position, and can determine the target decorative material that matches the basic material based on the matching of the features, and can automatically place the target decorative material in the corresponding position, thereby achieving the effect of intelligent adsorption of decorative materials and effectively improving the efficiency of multimedia editing.

[0042] The following combination Figure 2 , each step in steps S200~S208 and other optional steps are described in detail.

[0043] Step S200 When the imported basic material is received, the basic material is placed in the first track, where the basic material includes at least one of a picture, an audio, and a video.

[0044] The first track may be a video track in a multimedia editor, and is used to place basic materials imported by the user.

[0045] Specifically, the user can import the basic materials that he wants to edit, such as captured videos, pictures, recorded audio, etc., through the client. After receiving the basic materials imported by the user, the client puts the basic materials into the first track. When putting the basic materials into the first track, the basic materials can be put in the order of import or the order in which the user drags and drops them.

[0046] Step S202 , get the first feature of the basic material.

[0047] When the base material is an image, the first feature may include visual features (such as color, shape, texture, etc.) and semantic features (such as object category, scene description, etc.); when the base material is audio, the first feature may include acoustic features (such as frequency, amplitude, rhythm, etc.) and semantic features (such as speech content, emotion, etc.); when the base material is video, the first feature may be spatiotemporal features (such as image content, motion trajectory, etc.). Different analysis methods can be used to obtain the corresponding first features for different base materials. Of course, for base materials containing multiple types simultaneously, such as videos containing both images and audio, multiple analysis methods can be used to obtain the corresponding first features.

[0048] When the basic material corresponds to content of a certain length, such as a video, in order to facilitate the subsequent determination of the target location for placing the decorative material, when obtaining the first feature of the basic material, the first feature of a certain part of the content can be obtained.

[0049] In an exemplary embodiment, step S202 may further include: The first feature of the target content of the basic material in a specific time range is obtained.

[0050] The specific time range can be specified by the user, for example, the user selects a time range in the first track. Optionally, the specific time range can be a preset time interval, for example, a time interval of 10 seconds, and the first feature of the target content of the basic material in each time interval is obtained respectively. Alternatively, the time point corresponding to the specific object or specific time contained in the basic material can be analyzed first, and then a time range is defined as the specific time range based on the time point, and then the first feature of the target content of the basic material in the specific time range is obtained. For example, the key object in the video can be identified by target detection, and a time range is defined as the specific time range based on the time point when the key object appears.

[0051] In this embodiment, by obtaining the first feature of the target content of the basic material within a specific time range, it is convenient to subsequently determine the placement position of the decorative material.

[0052] In an optional embodiment, the target content includes frame images and audio, such as Figure 3 As shown, step S202 may include: Step S300: Determine content features of a frame image using computer vision technology.

[0053] Step S302: Determine the visual features of the frame image using image processing technology.

[0054] Step S304: Using audio processing technology to determine audio features of the audio.

[0055] Step S306: Obtain a first feature based on the content feature, the visual feature, and the audio feature.

[0056] Specifically, when using computer vision techniques to determine the content features of image frames, object detection models can be used to identify objects or people and their locations within the image frames, determining their category labels and bounding box coordinates. Scene classification models (such as convolutional neural networks) can also be used to identify the scene type of the image frames, such as "indoor" or "beach." Instance segmentation models can also be used to accurately segment the outlines of objects or people within the image frames. Optical flow can also be used to analyze the motion of objects between adjacent image frames to determine the type and intensity of the motion. When using image processing techniques to determine the visual features of image frames, RGB / HSV histograms and dominant color distribution can be calculated, or texture complexity can be extracted using local binary patterns or gray-level co-occurrence matrices. Pixel value distribution statistics can also be used to determine the contrast and brightness of the image frames. When using audio processing techniques to determine the audio features of audio, spectral envelope features can be extracted. Basic changes in audio can also be analyzed to determine the emotional state of the audio (e.g., determining excitement based on rising pitch). Energy analysis can also be used to identify sudden changes in volume. Automatic speech recognition can also be used to convert audio into text and extract its content and semantic features.

[0057] After obtaining the content features, visual features, and audio features, these three features can be processed and used as the first features of the target content. For example, the three features can be time-aligned, or two or three features can be concatenated. Of course, the three features can also be used directly as the first features of the target content. In this case, the first features contain features from three dimensions.

[0058] In this embodiment, by using computer vision technology to determine the content features of the frame image, using image processing technology to determine the visual features of the frame image, and using audio processing technology to determine the audio features of the audio, the first feature is obtained based on the content features, visual features and audio features, which can effectively perform feature analysis of multiple dimensions on the basic material, thereby improving the accuracy of subsequent determination of the corresponding decorative material.

[0059] Step S204 , obtain the second feature of the decoration material.

[0060] Decorative materials can include image, audio, and video materials. For example, image materials include stickers, text, filters, and special effects patterns; audio materials include music, sound effects, and voice narration; and video materials include transition animations and dynamic special effects. The second feature can be similar to the first feature. In practical applications, similar or identical analysis methods can be used to obtain the second feature of a decorative material.

[0061] In an optional embodiment, step S204 may include: The second feature of the decorative material is obtained through an artificial intelligence algorithm, where the second feature includes at least one of a content feature, a color feature, a shape feature, a dynamic feature, and a scene feature.

[0062] Content features can be the object, concept, or theme represented by the decorative material, such as a rose or a cartoon character. Image classification models (such as convolutional neural networks) can be used to identify content features in decorative materials. For materials containing text, optical character recognition can be used to extract text information, and natural language processing methods can be used to analyze the meaning of the text to obtain the content features of the decorative material. Color features can include the dominant hue, color saturation, and color contrast of the decorative material. The color histogram of the decorative material can be calculated to obtain color distribution characteristics. Clustering algorithms can also be used to cluster the colors in the decorative material to determine the dominant hue of the decorative material. Shape features can be the outline, structure, or geometric properties of the objects in the decorative material, such as edges and texture. Edge detection algorithms can be used to extract the outline of the objects in the decorative material, while the LBP algorithm can be used to extract the texture features of the decorative material. Dynamic features can be the properties of the decorative material that change over time, such as motion trajectory and speed. Optical flow methods can be used to estimate the motion vectors of pixels in the decorative material (such as video decorative material) to obtain the motion trajectory and speed information of the decorative material. The scene features can be the scene attributes described in the decoration material, such as indoor, natural scenery, urban architecture, etc. The scene classification model can be used to classify the decoration material to obtain the scene features of the decoration material.

[0063] It is understood that the artificial intelligence algorithm used above is only an example. In fact, other artificial intelligence algorithms can be used to implement it, as long as they can obtain the second feature of the decorative material. In addition, if some decorative materials do not have features, they can be ignored and only the features they do have can be extracted.

[0064] In this embodiment, the second feature of the decorative material is obtained by an artificial intelligence method, which can facilitate the subsequent matching of the decorative material with the basic material and improve the accuracy of the matching.

[0065] Step S206 Based on the first feature and the second feature, the decorative material is matched with the basic material, the decorative material matching the basic material is determined as the target decorative material, and the target position of the target decorative material is determined on the second track, and the second track is synchronized with the first track in time.

[0066] The second track may be a material track, which is synchronized with the first track in time.

[0067] Specifically, the similarity between the first feature and the second feature (such as cosine similarity, Euclidean distance, etc.) can be calculated to match the decorative material with the base material. When the similarity reaches a preset threshold, the decorative material is determined to match the base material. If multiple decorative materials match the base material, the decorative material with the highest similarity can be selected as the target decorative material. When determining the target position for the target decorative material on the second track, if the first feature is a characteristic of the target content of the base material within a specific time range, the target position for the target decorative material on the second track can be determined based on the specific time range. For example, if the specific time range is the target content of 10 to 15 seconds of the base material, the target position can be one of the positions between 10 and 15 seconds, such as the position at 10 seconds. Optionally, the target position can include a position range representing the duration of the decorative material. In this case, the target position can be, for example, the position interval between 10 and 15 seconds.

[0068] Step S208 , place the target decoration material into the target position.

[0069] After determining the target decorative material and target location, you can directly place the target decorative material into the target location to achieve the effect of automatic adsorption. To more clearly illustrate the matching of basic materials and decorative materials, the following are some application examples: (1) Stickers: Based on the frame images within a specific time range on the video track (the first track), stickers are automatically matched and placed in positions that match the content of the frame. For example, when a hilarious scene appears in the frame image of the basic material, a "laughing" sticker is automatically matched and placed.

[0070] (2) Music: Based on the semantic features (such as rhythm or emotion) corresponding to the frame images within a specific time range on the video track, music is automatically matched and placed in the position that matches the semantic features. For example, when there is intense action in the frame images of the basic material, music with a strong sense of rhythm is automatically matched and placed.

[0071] (3) Special effects: Based on the motion characteristics of the frame images within a specific time range on the video track, special effects are automatically matched and placed in positions that match the motion characteristics. For example, when fast motion occurs in the frame images of the basic material, a motion blur effect is automatically matched and placed.

[0072] (4) Text Adsorption: Based on the semantic features of the frame images within a specific time range on the video track, the text is automatically matched and placed in the position that matches the semantic features. For example, when a landscape appears in the frame image of the basic material, the text describing the landscape is automatically matched and placed.

[0073] (5) Filter Adsorption: Based on the visual characteristics (such as color and brightness) of the frame images within a specific time range on the video track, the filter is automatically matched and placed in a position that matches the visual characteristics. For example, when dark tones appear in the frame images of the basic material, a bright tones filter is automatically matched and placed.

[0074] In an optional embodiment, if Figure 4 As shown, the multimedia editing method in the embodiment of the present application may further include: Step S400: Obtain matching analysis results of target decoration materials.

[0075] Step S402: When the target decoration material is placed in the target position, the matching analysis result is visually displayed on the editing interface.

[0076] The matching analysis results may include, but are not limited to, content matching analysis results, color matching analysis results, and time matching analysis results. Specifically, they can be a sentence or a short summary description. For example, if the target decorative material is a sticker, the matching analysis result may be "the sticker matches the frame content." For another example, if the target decorative material is audio, the matching analysis result may be "the audio matches the corresponding rhythm of the frame content." Visual display methods may include, but are not limited to, color bars, progress bars, and text prompts.

[0077] Specifically, when the target decorative material is determined, the matching analysis result of the target decorative material can be obtained according to the matching situation of the target decorative material; when the target decorative material is placed in the target position of the second track, the matching analysis result is displayed in a preset area of ​​the current editing interface (for example, below the target position of the second track) in a visual manner, so that the user can clearly understand the reason why the target decorative material is placed.

[0078] In this embodiment, by obtaining the matching analysis result of the target decorative material and placing the target decorative material in the target position, the matching analysis result is visually displayed on the editing interface, so that the user can clearly understand the reason for placing the decorative material.

[0079] In an optional embodiment, the multimedia editing method of the embodiment of the present application may further include: When an adjustment instruction for the target decorative material is received, the position of the target decorative material on the second track is adjusted according to the adjustment instruction.

[0080] Adjustment instructions can be input via a mouse or touch, for example, the user drags a decorative material with the mouse to change its position on the second track; optionally, adjustment instructions can also be input via the keyboard, for example, by using the left and right arrow keys on the keyboard to move the target decorative material; adjustment instructions can also be input via voice, for example, the user inputs "move the sticker to the 5th second" via voice. Of course, the input of adjustment instructions can also be a combination of the above several methods, which can be specifically set according to actual needs and are not limited here.

[0081] For example, if the user feels that the placement position of the target decorative material "burst out laughing" is not ideal and the position should be adjusted 2 seconds further back, the target decorative material can be moved backward 2 seconds by using the right arrow key on the keyboard.

[0082] In practical applications, the corresponding logic or algorithm for determining the target position can also be continuously optimized according to the user's adjustment of the decorative material, so as to further reduce the user's adjustment operations on the decorative material and further improve the efficiency and accuracy of multimedia editing.

[0083] In this embodiment, by adjusting the position of the target decorative material on the second track according to the adjustment instruction, the placement of the decorative material can be further optimized according to the user's interaction, improving the accuracy and effect of multimedia editing.

[0084] Embodiment 2 Figure 5 The block diagram of the multimedia editing device according to Embodiment 2 of the present application is schematically shown. The device can be divided into one or more program modules. One or more program modules are stored in a storage medium and executed by one or more processors to complete the embodiments of the present application. The program modules referred to in the embodiments of the present application refer to a series of computer program instruction segments that can complete specific functions. The following description will specifically introduce the functions of each program module in this embodiment. As Figure 5 shown, the device 500 may include: a first placement module 510, a first acquisition module 520, a second acquisition module 530, a determination module 540, and a second placement module 550, where: The first placement module 510 is configured to place the base material in the first track when receiving the imported base material, where the base material includes at least one of pictures, audio, and video; The first acquisition module 520 is configured to acquire the first feature of the base material; The second acquisition module 530 is configured to acquire the second feature of the decorative material; a determination module 540 configured to match a decorative material with the base material based on the first feature and the second feature, determine the decorative material that matches the base material as a target decorative material, and determine a target position for placing the target decorative material on a second track, where the second track is synchronized with the first track in time; The second placing module 550 is used to place the target decoration material into the target position.

[0085] In an optional embodiment, the first obtaining module 520 is further configured to: A first feature of the target content of the basic material within a specific time range is obtained.

[0086] In an optional embodiment, the target content includes frame images and audio; Correspondingly, the first acquisition module 520 is further configured to: Determining content features of the frame image using computer vision technology; Determining visual features of the frame image using image processing technology; Determining audio characteristics of the audio using audio processing technology; The first feature is obtained based on the content feature, the visual feature, and the audio feature.

[0087] In an optional embodiment, the second obtaining module 530 is further configured to: The second feature of the decorative material is obtained through an artificial intelligence algorithm, where the second feature includes at least one of a content feature, a color feature, a shape feature, a dynamic feature, and a scene feature.

[0088] In an optional embodiment, the apparatus 500 is further configured to: Obtaining a matching analysis result of the target decoration material; When the target decoration material is placed in the target position, the matching analysis result is visually displayed on an editing interface.

[0089] In an optional embodiment, the apparatus 500 is further configured to: When an adjustment instruction for the target decoration material is received, the position of the target decoration material on the second track is adjusted according to the adjustment instruction.

[0090] Example 3 Figure 6The following schematically shows a hardware architecture diagram of a computer device 10000 suitable for implementing a multimedia editing method according to the third embodiment of the present application. In some embodiments, the computer device 10000 may be a terminal device such as a smart phone, a wearable device, a tablet computer, a personal computer, a vehicle-mounted terminal, a game console, a virtual device, a workbench, a digital assistant, a set-top box, a robot, etc. In other embodiments, the computer device 10000 may be a rack server, a blade server, a tower server, or a cabinet server (including an independent server or a server cluster composed of multiple servers), etc. Figure 6 As shown, the computer device 10000 includes but is not limited to: a memory 10010, a processor 10020, and a network interface 10030 that can communicate with each other via a system bus. Memory 10010 includes at least one type of computer-readable storage medium, including flash memory, a hard disk, a multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, a magnetic disk, an optical disk, and the like. In some embodiments, memory 10010 may be an internal storage module of computer device 10000, such as a hard disk or memory of computer device 10000. In other embodiments, memory 10010 may also be an external storage device of computer device 10000, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, and the like equipped on computer device 10000. Of course, memory 10010 may also include both internal storage modules and external storage devices of computer device 10000. In this embodiment, the memory 10010 is generally used to store an operating system and various application software installed on the computer device 10000, such as program codes of a multimedia editing method, etc. In addition, the memory 10010 can also be used to temporarily store various data that has been output or is to be output.

[0091] In some embodiments, processor 10020 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other chips. Processor 10020 is typically used to control the overall operation of computer device 10000, such as performing control and processing related to data exchange or communication with computer device 10000. In this embodiment, processor 10020 is used to execute program code stored in memory 10010 or process data.

[0092] Network interface 10030 may include a wireless network interface or a wired network interface. Network interface 10030 is typically used to establish a communication link between computer device 10000 and other computer devices. For example, network interface 10030 is used to connect computer device 10000 to an external terminal via a network, establishing a data transmission channel and a communication link between computer device 10000 and the external terminal. The network may be a wireless or wired network such as an intranet, the Internet, the Global System of Mobile Communications (GSM), Wideband Code Division Multiple Access (WCDMA), a 4G network, a 5G network, Bluetooth, or Wi-Fi.

[0093] It should be pointed out that Figure 6 Only a computer device having components 10010 - 10030 is shown, but it should be understood that implementing all of the shown components is not a requirement, and more or fewer components may alternatively be implemented.

[0094] In this embodiment, the multimedia editing method stored in the memory 10010 can also be divided into one or more program modules and executed by one or more processors (such as processor 10020) to complete the embodiment of the present application.

[0095] Example 4 An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the multimedia editing method in the embodiment are implemented.

[0096] In this embodiment, computer-readable storage media include flash memory, hard disks, multimedia cards, card-type memories (e.g., SD or DX memories), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, magnetic disks, optical disks, and the like. In some embodiments, the computer-readable storage medium may be an internal storage unit of a computer device, such as the computer device's hard disk or memory. In other embodiments, the computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, and the like. Of course, the computer-readable storage medium may also include both the internal storage unit and external storage devices of the computer device. In this embodiment, the computer-readable storage medium is typically used to store the operating system and various application software installed on the computer device, such as the program code of the multimedia editing method described in the embodiment. In addition, the computer-readable storage medium may also be used to temporarily store various types of data that has been output or is about to be output.

[0097] Example 5 An embodiment of the present application further provides a computer program product, including a computer program, which implements the method in the above embodiment when executed by a processor.

[0098] Obviously, those skilled in the art should understand that the modules or steps of the above-mentioned embodiments of the present application can be implemented using general-purpose computer devices. They can be concentrated on a single computer device or distributed on a network composed of multiple computer devices. Alternatively, they can be implemented using program codes executable by the computer device, so that they can be stored in a storage device and executed by the computer device. In some cases, the steps shown or described can be performed in a different order than herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0099] It should be noted that the above are only preferred embodiments of the present application and do not limit the scope of patent protection of the present application. Any equivalent structure or equivalent process transformation made using the contents of the description and drawings of this application, or directly or indirectly applied in other related technical fields, are also included in the scope of patent protection of the present application.

Claims

1. A multimedia editing method, characterized in that: The method comprises: Upon receiving the imported basic material, placing the basic material into the first track, the basic material comprising at least one of a picture, audio, and video; Obtaining a first feature of the basic material; Obtaining the second characteristic of the decoration material; matching the decorative material with the base material based on the first feature and the second feature, determining the decorative material that matches the base material as a target decorative material, and determining a target position for placing the target decorative material on a second track, where the second track is synchronized with the first track in time; The target decoration material is placed in the target position.

2. The method according to claim 1, characterized in that The obtaining of the first feature of the basic material includes: A first feature of the target content of the basic material within a specific time range is obtained.

3. The method according to claim 2, characterized in that The target content includes frame images and audio; Correspondingly, the step of obtaining the first feature of the target content of the basic material within a specific time range includes: Determining content features of the frame image using computer vision technology; Determining visual features of the frame image using image processing technology; Determining audio characteristics of the audio using audio processing technology; The first feature is obtained based on the content feature, the visual feature, and the audio feature.

4. The method according to claim 1, wherein The second feature of obtaining the decoration material includes: The second feature of the decorative material is obtained through an artificial intelligence algorithm, where the second feature includes at least one of a content feature, a color feature, a shape feature, a dynamic feature, and a scene feature.

5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Obtaining a matching analysis result of the target decoration material; When the target decoration material is placed in the target position, the matching analysis result is visually displayed on an editing interface.

6. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: When an adjustment instruction for the target decoration material is received, the position of the target decoration material on the second track is adjusted according to the adjustment instruction.

7. A multimedia editing device, characterized in that: The device comprises: A first inserting module is configured to insert the imported basic material into the first track upon receiving the imported basic material, wherein the basic material includes at least one of a picture, audio, and video; A first acquisition module, configured to acquire a first feature of the basic material; A second acquisition module, used to acquire a second feature of the decoration material; a determination module, configured to match a decorative material with the base material based on the first feature and the second feature, determine the decorative material that matches the base material as a target decorative material, and determine a target position for placing the target decorative material on a second track, wherein the second track is synchronized with the first track in time; The second placing module is used to place the target decoration material into the target position.

8. A computer device, characterized in that: include: at least one processor; and a memory communicatively coupled to the at least one processor; wherein: The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, which, when executed by a processor, implement the method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images