Multimedia file editing method, device and computing equipment

Abstract

This invention relates to the field of video processing technology, and discloses a multimedia file editing method, apparatus, computing device, and computer-readable storage medium. The method includes: determining a target list matching the content type of a currently playing multimedia file; and, in response to a user's selection of a target from the target list, editing the currently playing multimedia file to obtain edited content. Through this method, this invention achieves automatic generation of edited content, enhancing the user experience.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of video processing technology, and specifically to a multimedia file editing method, device, computing equipment and computer-readable storage medium. Background Art

[0002] At present, for multimedia file editing, such as video editing, both professional media and user content production require real-time video or photo shooting, or editing of offline stored videos or photos. However, the existing multimedia file editing methods have low social real-time and fun, and have high requirements for high-quality content production. The production of video content requires a high degree of video processing expertise, or relies on professional video processing tools, which is not friendly to most ordinary non-professional users and has low processing efficiency. Although existing content production applications have been developed and embedded with some video rendering skills, because they are batch production for ordinary users, the scalability and mining of user self-production and creativity are not high. Summary of the invention

[0003] In view of the above problems, embodiments of the present invention provide a multimedia file clipping method, apparatus, computing device and computer-readable storage medium to solve the problem of low efficiency of multimedia file clipping in the prior art.

[0004] According to one aspect of an embodiment of the present invention, a method for clipping a multimedia file is provided, the method comprising:

[0005] Determine, according to the content type of the multimedia file currently being played, a target list matching the content type;

[0006] In response to a user's selection operation on a target in the target list, the currently played multimedia file is clipped to obtain clipped content.

[0007] In an optional manner, determining a target list that matches the content type based on the content type of the multimedia file currently being played includes: matching the content type of the multimedia file currently being played in a type-target table in a database to obtain a target list that matches the content type; the type-target list in the database is a table of content type-corresponding targets obtained by pre-identifying targets in different types of multimedia file samples.

[0008] In an optional manner, the method of matching a type-target list in a database according to the content type of the multimedia file currently being played to obtain a target list matching the content type includes: matching a type-target list in a database according to the content type of the multimedia file currently being played to obtain each target corresponding to the multimedia file currently being played and the probabilities of the targets; and sorting the targets according to the probabilities of the targets to obtain a target list matching the content type.

[0009] In an optional manner, before determining a target list matching the content type based on the content type of the multimedia file currently being played, the method includes: acquiring the content type of a multimedia file sample; identifying the multimedia file sample to obtain a target corresponding to the multimedia file sample; and obtaining a type-target relationship based on the content type of the multimedia file sample and the target corresponding to the multimedia file sample.

[0010] In an optional manner, after identifying the multimedia file sample and obtaining the target corresponding to the multimedia file sample, the method includes: determining the number of times each target appears and the total number of times all targets appear in each type of multimedia file sample; and determining the probability of the relationship between the type and the target based on the number of times each target appears and the total number of times all targets appear.

[0011] In an optional manner, in response to the user's selection operation on a target in the target list, the currently playing multimedia file is clipped, and before obtaining the clipped content, it includes: displaying the target list matching the content type to the user; and receiving the user's selection of the target in the target list.

[0012] In an optional manner, in response to the user's selection operation on a target in the target list, the currently playing multimedia file is clipped to obtain clipped content, including: determining the position of the target selected by the user in the multimedia file; and clipping the currently playing multimedia file according to the position to obtain clipped content.

[0013] According to another aspect of an embodiment of the present invention, there is provided a multimedia file editing device, comprising:

[0014] A determination module, configured to determine a target list matching the content type according to the content type of the multimedia file currently being played;

[0015] The clipping module is used to clip the currently played multimedia file in response to the user's selection operation on the target in the target list to obtain clipped content.

[0016] According to another aspect of an embodiment of the present invention, a computing device is provided, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; the memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the operation of the multimedia file editing method.

[0017] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, wherein the storage medium stores at least one executable instruction, and when the executable instruction is executed on a computing device, the computing device executes the operation of the multimedia file editing method.

[0018] The embodiment of the present invention determines a target list matching the content type according to the content type of the multimedia file currently being played, and in response to a user's selection operation on a target in the target list, clips the multimedia file currently being played to obtain clipped content. The embodiment of the present invention can match the content type in real time and efficiently recommend clipped motion content to the user. During video playback, a pre-acquired motion list is provided to the user, and the multimedia file is clipped according to the motion selected by the user. The clipped content is automatically generated in the background, thereby realizing a content self-production social system, improving user experience, and increasing user traffic and stickiness.

[0019] The above description is only an overview of the technical solution of the embodiment of the present invention. In order to more clearly understand the technical means of the embodiment of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiment of the present invention more obvious and easy to understand, the specific implementation mode of the present invention is specifically listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings are only used to illustrate the embodiments and are not to be considered as limiting the present invention. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings. In the accompanying drawings:

[0021] Figure 1 A schematic diagram showing a flow chart of a multimedia file editing method provided by an embodiment of the present invention;

[0022] Figure 2 A schematic diagram showing the structure of a multimedia file editing device provided by an embodiment of the present invention is shown;

[0023] Figure 3 A schematic diagram of the structure of a computing device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0024] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0025] With the advancement of technology, the consumption of multimedia files, such as short video content, is becoming more and more commercialized, and the scenarios in which they are needed are becoming more and more extensive, including individual consumers watching or making them to pass the time, internal corporate promotions in companies, and professional producers making professional and interesting content to generate traffic.

[0026] From the previous professional media production to the current popularization of user-oriented content production applications, it is currently widely used. However, whether it is professional media or user content production, its use is limited. It requires real-time video or photo shooting, or editing of offline stored videos or photos, which reduces social real-time and fun. In addition, the requirements for high-quality content production are also relatively high. Looking at professional producers or more popular individual user producers, the production of video content requires a high degree of video processing expertise, or relies on professional video processing tools, which is not friendly to most ordinary non-professional users and has low production efficiency. Although existing content production applications have been developed and embedded with some video rendering skills, because it is batch production for ordinary users, the scalability and ability to tap into users' self-production and creativity are not high.

[0027] Currently, there are mainly three methods: 1. Extract video segments according to the starting time point, the ending time point and the corresponding video ID, and return the video address of the video segment file. (A method and device for real-time editing of online live video) 2. Transcode the live stream video in RTMP format into m3u8 format through video processing tools, and use m3u8 files for editing. (A method and device for real-time editing of live streaming video) 3. Obtain the picture frame of the live video stream for image recognition, and edit the target video segment from the live video stream according to the editing position. (Live multimedia file editing method, device and electronic device).

[0028] In view of the problems of existing multimedia file editing and the limitations of application scenarios, the present application proposes a multimedia file editing method for application in a self-production social system using real-time editing of content.

[0029] Figure 1 The flowchart of the multimedia file editing method provided by the embodiment of the present invention is shown, and the method is executed by a computing device. The computing device can be a computer device, a terminal device, such as a server, a video player, a reader, etc. Figure 1 As shown, the method comprises the following steps:

[0030] Step 110: According to the content type of the multimedia file currently being played, a target list matching the content type is determined.

[0031] Among them, the currently played multimedia file can be a video, audio or text, and the embodiment of the present invention does not make specific restrictions. The target in the embodiment of the present invention refers to the target existing in the content of the multimedia file. Among them, if the multimedia file is audio or text, the target can be a person, scene, etc., and the corresponding person, scene and other targets can be generated by identifying the text content corresponding to the audio. If the multimedia file is a video, the target can be an action, a person, a scene, etc., or a combination of actions, people, scenes, etc., and those skilled in the art can make corresponding settings according to the specific scene, and the embodiment of the present invention does not make specific restrictions.

[0032] The embodiments of the present invention are described below by taking a multimedia file being a video as an example.

[0033] In the embodiment of the present invention, a type-target list is pre-set in the database, and the type-target list in the database is a table of content type corresponding targets obtained by pre-identifying targets in different types of multimedia file samples. The specific generation process of the type-target list includes:

[0034] Various targets corresponding to various content types are set, which can be set manually or by using an algorithm to identify targets of multimedia file samples to obtain a target list corresponding to the multimedia file samples. The process of identifying targets of video samples by using an algorithm to obtain a target list corresponding to the video samples is as follows: obtaining the content type of the video sample; identifying the multimedia file sample to obtain the target corresponding to the multimedia file sample; and obtaining the type-target relationship based on the content type of the multimedia file sample and the target corresponding to the multimedia file sample.

[0035] Specifically, taking the target as an action, take the existing video resources in the media resource library, randomly extract N video samples according to the preset statistical sampling rules, and obtain the type (content type) of each video. The preset statistical sampling rules can be: set the video information of the video sample, and simplify the video information query conditions to [video ID-video type level (video subdivision level)-video type], where video type level-video type represents the video category. In one embodiment of the present invention, the sampling statistical sampling rules are: according to the corresponding video type level-video type classification, obtain layered data, and then randomly extract a certain number of video IDs for each layer of video type level-video type. Among them, the statistical sampling rules can also be combined with specific business requirements and business indicators. For example, random extraction can be replaced by sorting from high to low according to indicators such as video playback volume, completion rate, and ratings, and the video samples used in the construction process are limited to videos that general users are interested in, simplifying performance and improving hit rates. Extracting a certain number of video IDs means that in order to obtain N video samples, the number of videos to be extracted for each type (content type) can be evenly distributed according to the total number of N / type. If the total number of types is 10, the number of videos to be extracted for each type should be N / 10. However, considering that the total number of videos corresponding to each type is unevenly distributed, and not every type can extract N / 10 videos, and the total number of videos of some types is less than N / 10, the number of extracted videos can also be reasonably calculated according to the total distribution of videos corresponding to the type, that is, extracted in proportion, which is N*(the proportion of the total number of videos corresponding to a certain type to the total number of videos). For example, the total number of videos corresponding to a certain type is ntype, and the total number of videos in the media resource library is Ntotal, then the proportion of the total number of videos corresponding to this type to the total number of videos is ntype / Ntotal, and the corresponding number of videos to be extracted should be N*(ntype / Ntotal). In addition, considering the impact of the type subdivision level (video type level) on the sampled video, some first-level types are too large, resulting in too large a proportion and over-sampling. Therefore, when extracting videos, only the lowest level type is extracted. For example, if the type is subdivided into three levels, it is extracted according to the three levels. Although the remaining levels of type do not participate in the sampling extraction, the information is saved in the information data corresponding to the video ID for subsequent processing. Then, the video is identified through the algorithm to obtain its motion list.Among them, the same video may have more than one type, and there may be several levels of type under the same first-level type. Therefore, according to the Cartesian product (Cartesian product refers to the Cartesian product of two sets X and Y in mathematics, expressed as X×Y, the first object is a member of X and the second object is one of all possible ordered pairs of Y), multiple types (content types) and motions are associated, and the type-motion relationship in N video samples is deduplicated and then stored in the type-motion table. Among them, deduplication includes video deduplication and type-motion relationship deduplication. For video deduplication, if a video has only one independent type, there is no video deduplication. However, the same video may have more than one type, and there may be several levels of type under the same first-level type. Based on this, according to the above sampling and video extraction rules, there are duplicate videos in the N video samples. For the video sample to perform action recognition through the algorithm and obtain its motion list, that is, before extracting motion from the video, deduplication of the video can reduce its processing difficulty. For the type-motion relationship deduplication, because the extracted relationships from different videos may be repeated, for example, love-kissing may appear in many videos, the extracted relationships will be repeated accordingly. However, only one record is needed to store the relationship itself, that is, after deduplication of the subscript combination of the probability matrix, only one group without duplication remains. Among them, whether the content type is divided into several levels depends on the specific situation. For example, type = ball game, ball game is its first-level type, and it can be divided into second-level types such as football, basketball, badminton, etc. For example, the second-level type football can be further divided into third-level types such as football league, football championship, and World Cup football.

[0036] In an embodiment of the present invention, after the multimedia file sample is identified and the target corresponding to the multimedia file sample is obtained, the probability of the relationship between the type and the target is also determined. Including: determining the number of times each target appears and the total number of times all targets appear in each type of multimedia file sample; determining the probability of the relationship between the type and the target based on the number of times each target appears and the total number of times all targets appear. Among them, a content type may correspond to multiple targets, and each target has a corresponding probability in the content type. The probability of the relationship between the type and the target can be represented by a probability value probability matrix, and the probability of the extracted type-motion relationship appearing in N multimedia file samples is assigned to obtain a probability value probability matrix. Specifically, it is determined according to the motion ratio corresponding to each type, and 0.24 represents the proportion of motion hugs in a specific type, i.e., love. For example, if we filter out n videos of type = love, there are 5 types of motion in total, and the number of times these 5 types of motion occur is 100, then probability [love, kiss] = 0.31 means kissing occurs 31 times in 100 actions, and probability [love, hug] = 0.24 means hugging occurs 24 times in 100 actions. Among them, the probability value of the type-target relationship that does not exist is 0. For example, probability [love, goal] = 0.00. Here, if the target only considers actions, such as kissing, the sum of the probability vectors corresponding to a certain type is 1. If motion considers scenes in addition to actions, such as sweetness, the sum of the probability vectors may be greater than 1. Among them, the probability value probability is set as a two-dimensional matrix. In order to facilitate the search, calculation and update, the matrix is ​​set as an equal-rank matrix with variable dimensions, and the dimension is the number of types * the number of motions, that is, [number of types, number of motions]. Because the number of types and the number of motions will be updated according to the training iteration, the matrix dimension is variable, and the value inside is set to double type, and the default value is 0.00.

[0037] In the embodiment of the present invention, the type-target list is also regularly updated and optimized. For example, considering that the original content type of the multimedia file in the media resource library may not be the most refined content type, the target list found by association is not the most reasonable. Therefore, combined with the concept of type segmentation mentioned above, the target can be reprocessed, that is, the target is matched according to the high-level content type and the low-level content type. Whether the type is divided into several levels depends on the specific scene. For example, when the multimedia file is a video, type = ball game, ball game is its first-level type, and it supports being divided into second-level types such as football, basketball, and badminton. For example, the second-level type football can be further divided into third-level types such as football league, football championship, and World Cup football. For example, the original type is a ball game, which may contain motions such as goals, yellow cards, kick-offs, and wrestling. According to the threshold, three motions of goals, yellow cards, and kick-offs are screened. In this scenario, the ball entity in the video is considered to be positioned and identified by the video image, and the ball entity is judged to which type it belongs. This judgment is the positive feedback of the video refinement type. According to different types of ball entities, the motion corresponding to the low-level type will be associated, which is more refined. For example, when a football is identified, its corresponding motion associations from high to low may be juggling, kicking, catching, and intercepting. For example, if juggling and kicking are above the threshold, the motion matched by the original first-level type is expanded through the low-level type, reducing the possible limitations of the first-level type. In addition, for some cross-action scenes, that is, only the motion scene corresponding to the type is limited or the action is limited, the action is expanded according to the scene, or the action is expanded according to the scene, such as there are many football actions in the warmth, or there are some warm hugs in the football game. The key information in the text can be extracted by combining the video introduction and subtitle text information, and the expanded type-target table can be matched.

[0038] Through the above method, the correspondence and probability between the content type and the target are obtained, thereby obtaining a type-target list. After the type-target table is output according to the algorithm training, it is stored in the database. As long as the content type is obtained, the target list of the corresponding video can be obtained by querying the type-target table. In any subsequent video, whether it is live broadcast, playback, cache, etc., there is no need to fully identify the motion through the algorithm. It only needs to update the iterative type-motion table, and at the algorithm level, the motion list initialized by multimedia file sample identification is more efficient than the two-step iteration of video frame / segment identification and motion positioning of video frame / segment in real-time positioning algorithm, so the efficiency experience is improved.

[0039] Therefore, after obtaining the content type of the multimedia file currently being played, the embodiment of the present invention matches the type-target table in the database according to the content type of the multimedia file currently being played, thereby obtaining a target list matching the content type. The target list matching the content type includes each target corresponding to the multimedia file currently being played and the probability of the target. Specifically, according to the content type of the multimedia file currently being played, the type-target list in the database is matched to obtain each target corresponding to the multimedia file currently being played and the probability of the target, and the targets are sorted according to the probability of the target to obtain the target list matching the content type. Among them, the targets in the target list matching the content type can be arranged from large to small according to the probability. For example, according to the content type of the multimedia file currently being played by the client, the type-motion table in the database is matched, and its initial value list is taken out, such as type = love, which may include motion such as kissing, hugging, sweet pet, sugar, etc.; type = ball game, which may include motion such as goal, yellow card, kick-off, wrestling, etc., are sorted from large to small in the list according to the probability value probability.

[0040] Step 120: In response to the user's selection operation on a target in the target list, the currently played multimedia file is clipped to obtain clipped content.

[0041] In an embodiment of the present invention, in response to a user's selection operation on a target in the target list, the currently playing multimedia file is clipped to obtain the clipped content, and the target list matching the content type is also displayed to the user, and the user's selection of the target in the target list is received. In an embodiment of the present invention, the targets in the target list matching the content type whose probability values ​​are greater than a preset threshold can be displayed to the user. The user selects the target to be clipped, and after the client receives the user's selection operation, during the video viewing process, the currently playing multimedia file is processed in the background according to the user's selection operation to generate clipped content, thereby not affecting the user's viewing progress. The clipped content is a clipped video. If the currently playing multimedia file is a live video, after the user pauses the live video, the background clipping runs normally, thereby not affecting the scene segments missed when the user continues to watch.

[0042] In an embodiment of the present invention, in response to a user's selection operation of a target in the target list, the position of the target selected by the user in the multimedia file is determined, and the multimedia file currently being played is clipped according to the position to obtain clipped content. The position of the target selected by the user in the video can be identified by a preset recognition algorithm, such as image recognition. In one embodiment of the present invention, when the target is a scene, the type of detailed action scene in the video can also be identified based on the changes in the user's facial expressions when watching. For example, if the user laughs, the video scene is identified as funny, and if the user cries, it is identified as touching.

[0043] The target in the target list to be displayed to the user can also be determined in combination with user characteristics. The user characteristics are generated based on historical user habits, for example, based on historical selection of targets, so as to recommend targets to the user better, faster and more accurately.

[0044] Among them, there may be multiple targets in the multimedia file currently being played at different times, so it is necessary to perform multiple clips on the targets in the multimedia file currently being played. After obtaining the content of each clip, the clip content can be integrated. For example, for a football game, the identified stars can be classified to form a star-action combination classification clip; for a film and television drama, the actors appearing in the drama can be identified according to the cast list, and the protagonist role can be clipped to form a role-action combination classification clip, etc.

[0045] In an embodiment of the present invention, after obtaining the clipped content, the clipped content or a list of clipped content can be displayed to the user. For example, the clipped content or a list of clipped content can be displayed to the user when the user pauses the currently playing multimedia file or when the currently playing multimedia file ends. The user can choose and then make it on his own, for example, video processing such as face changing, scene changing, and voice changing can be added to obtain the user's produced content. Receive the user's sharing or publishing instructions, and share or publish the corresponding user's produced content. Among them, after sharing or publishing the corresponding user's produced content, it is also possible to receive other users' forwarding, commenting or liking operations on the user's produced content, thereby increasing the user's interactive activity, improving user stickiness and the social attributes of the application.

[0046] Among them, the method also includes: showing online activities to users so that users can upload videos processed by the users and participate in online activities, thereby mobilizing users' production enthusiasm, improving the quality of self-produced content, and forming a virtuous ecosystem.

[0047] In an embodiment of the present invention, it is also possible to perform editing and generate editing content according to the user's operation on the video. For example, when the user takes a screenshot of a scene or an image frame of interest in the currently playing multimedia file during video viewing, the video frame that is captured is edited to generate editing content. Because during video playback, the user's screenshot operation may have a reaction time, and the longer the user's reaction time, the farther the actual captured frame is from the expected frame, and the larger the scene offset of the gif that is to be generated. In order to eliminate the difference between the captured frame and the expected frame caused by the reaction time and improve the accuracy of the captured video frame, several frames before the actual captured frame can be saved for selection, or when the user triggers the screenshot, a frame selection of several frames before and after is given, and the user independently selects a specific frame to generate a gif (editing content). If the user chooses to generate a gif, consider sending the gif directly in the live comment to share and interact; if the application performance supports it, the above real-time edited video can also be shared in the comment area through video fingerprints or video playback thumbnails or video QR codes. Of course, the sharing function is not limited to the comment area, and can also be shared across apps and social applications. On the basis of forming an interactive ecosystem in the comment area for video clips and GIF dynamics, it can be set up to support users in the comment area (chat room) to click on the avatar or user name to view the user information. The information includes all content produced by the user through real-time editing, including but not limited to content based on the current game or the video itself. Of course, you can also customize the visibility of the content, and display some content through visibility for users you follow, friends, or strangers. The premise for users to view each other's clipped content is that the user has published the produced content, and the client supports users to publish content. With copyright permission, the content can also be saved locally. On the basis of viewing, the functions of forwarding, commenting, and liking can also be expanded.

[0048] In an embodiment of the present invention, user comments can also be received. Specifically, the user can also start the comment entry through voice, convert voice to text, and generate videos or screenshots through expression / voice synthesis. The specific rules are: through voice emotion and text emotion matching + picture / gif matching, weighted matching, pictures / gifs are also matched according to the filtering algorithm to store the first N items that did not appear in the database. For example, the voice emotion recognition is excitement, the text emotion recognition is positive, and the picture / gif emotion matching is an excited and positive type (according to weighted matching, it may not be an exact match). According to the user's comments, the video frame of the currently playing multimedia file is located, and the video frame is edited to generate the editing content. Among them, the time point when the user thinks the wonderful comment is sent can be determined according to the user's comments, so as to locate the video frame.

[0049] In the embodiment of the present invention, in order to improve performance, a large amount of cached data is automatically cleaned up regularly. According to the number of likes on the comments and the similarity of the video and the picture, if it exceeds the threshold, the local cache is directly cleaned up or selectively retained.

[0050] The embodiment of the present invention determines a target list matching the content type according to the content type of the multimedia file currently being played, and in response to a user's selection operation on a target in the target list, clips the multimedia file currently being played to obtain clipped content. The embodiment of the present invention can match the content type in real time and efficiently recommend clipped action content to the user. During video playback, a pre-acquired motion list is provided to the user, and the multimedia file is clipped according to the motion selected by the user. The clip is automatically generated in the background, thereby realizing a content self-production social system, improving user experience, and increasing user traffic and stickiness.

[0051] Figure 2 FIG. 1 is a schematic diagram showing the structure of a multimedia file editing device provided by an embodiment of the present invention. Figure 2 As shown, the device 200 includes: a determination module 210 and a clipping module 220.

[0052] The determination module 210 is used to determine a target list matching the content type according to the content type of the multimedia file currently being played.

[0053] The clipping module 220 is used to clip the currently played multimedia file to obtain clipped content in response to the user's selection operation on the target in the target list.

[0054] In an optional manner, determining a target list that matches the content type based on the content type of the multimedia file currently being played includes: matching the content type of the multimedia file currently being played in a type-target table in a database to obtain a target list that matches the content type; the type-target list in the database is a table of content type-corresponding targets obtained by pre-identifying targets in different types of multimedia file samples.

[0055] In an optional manner, the method of matching a type-target list in a database according to the content type of the multimedia file currently being played to obtain a target list matching the content type includes: matching a type-target list in a database according to the content type of the multimedia file currently being played to obtain each target corresponding to the multimedia file currently being played and the probabilities of the targets; and sorting the targets according to the probabilities of the targets to obtain a target list matching the content type.

[0056] In an optional manner, before determining a target list matching the content type based on the content type of the multimedia file currently being played, the method includes: acquiring the content type of a multimedia file sample; identifying the multimedia file sample to obtain a target corresponding to the multimedia file sample; and obtaining a type-target relationship based on the content type of the multimedia file sample and the target corresponding to the multimedia file sample.

[0057] In an optional manner, after identifying the multimedia file sample and obtaining the target corresponding to the multimedia file sample, the method includes: determining the number of times each target appears and the total number of times all targets appear in each type of multimedia file sample; and determining the probability of the relationship between the type and the target based on the number of times each target appears and the total number of times all targets appear.

[0058] In an optional manner, in response to the user's selection operation on a target in the target list, the currently playing multimedia file is clipped, and before obtaining the clipped content, it includes: displaying the target list matching the content type to the user; and receiving the user's selection of the target in the target list.

[0059] In an optional manner, in response to the user's selection operation on a target in the target list, the currently playing multimedia file is clipped to obtain clipped content, including: determining the position of the target selected by the user in the multimedia file; and clipping the currently playing multimedia file according to the position to obtain clipped content.

[0060] The working process of the multimedia file editing device of the embodiment of the present invention is substantially consistent with the method steps of the multimedia file editing method described above, and will not be repeated here.

[0061] The embodiment of the present invention determines a target list matching the content type according to the content type of the multimedia file currently being played, and in response to a user's selection operation on a target in the target list, clips the multimedia file currently being played to obtain clipped content. The embodiment of the present invention can match the content type in real time and efficiently recommend clipped motion content to the user. During video playback, a pre-acquired motion list is provided to the user, and the multimedia file is clipped according to the motion selected by the user. The clip is automatically generated in the background, thereby realizing a content self-production social system, improving the user's video consumption experience, and increasing user traffic and stickiness.

[0062] Figure 3 A schematic diagram of the structure of a computing device provided by an embodiment of the present invention is shown, and the specific embodiment of the present invention does not limit the specific implementation of the computing device.

[0063] like Figure 3As shown, the computing device may include: a processor (processor) 302 , a communication interface (Communications Interface) 304 , a memory (memory) 306 , and a communication bus 308 .

[0064] The processor 302, the communication interface 304, and the memory 306 communicate with each other via the communication bus 308. The communication interface 304 is used to communicate with other devices such as a client or other server network elements. The processor 302 is used to execute the program 310, which can specifically execute the relevant steps in the above-mentioned embodiment of the method for multimedia file editing.

[0065] Specifically, the program 310 may include program code including computer executable instructions.

[0066] The processor 302 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. The one or more processors included in the computing device may be processors of the same type, such as one or more CPUs, or may be processors of different types, such as one or more CPUs and one or more ASICs.

[0067] The memory 306 is used to store the program 310. The memory 306 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0068] The program 310 may be specifically invoked by the processor 302 to cause the computing device to perform the following operations:

[0069] Determine, according to the content type of the multimedia file currently being played, a target list matching the content type;

[0070] In response to a user's selection operation on a target in the target list, the currently played multimedia file is clipped to obtain clipped content.

[0071] In an optional manner, determining a target list that matches the content type based on the content type of the multimedia file currently being played includes: matching the content type of the multimedia file currently being played in a type-target table in a database to obtain a target list that matches the content type; the type-target list in the database is a table of content type-corresponding targets obtained by pre-identifying targets in different types of multimedia file samples.

[0072] In an optional manner, the method of matching a type-target list in a database according to the content type of the multimedia file currently being played to obtain a target list matching the content type includes: matching a type-target list in a database according to the content type of the multimedia file currently being played to obtain each target corresponding to the multimedia file currently being played and the probabilities of the targets; and sorting the targets according to the probabilities of the targets to obtain a target list matching the content type.

[0073] In an optional manner, before determining a target list matching the content type based on the content type of the multimedia file currently being played, the method includes: acquiring the content type of a multimedia file sample; identifying the multimedia file sample to obtain a target corresponding to the multimedia file sample; and obtaining a type-target relationship based on the content type of the multimedia file sample and the target corresponding to the multimedia file sample.

[0074] In an optional manner, after identifying the multimedia file sample and obtaining the target corresponding to the multimedia file sample, the method includes: determining the number of times each target appears and the total number of times all targets appear in each type of multimedia file sample; and determining the probability of the relationship between the type and the target based on the number of times each target appears and the total number of times all targets appear.

[0075] In an optional manner, in response to the user's selection operation on a target in the target list, the currently playing multimedia file is clipped, and before obtaining the clipped content, it includes: displaying the target list matching the content type to the user; and receiving the user's selection of the target in the target list.

[0076] In an optional manner, in response to the user's selection operation on a target in the target list, the currently playing multimedia file is clipped to obtain clipped content, including: determining the position of the target selected by the user in the multimedia file; and clipping the currently playing multimedia file according to the position to obtain clipped content.

[0077] The working process of the multimedia file editing device of the embodiment of the present invention is substantially consistent with the method steps of the multimedia file editing method described above, and will not be repeated here.

[0078] The embodiment of the present invention determines a target list matching the content type according to the content type of the multimedia file currently being played, and in response to a user's selection operation on a target in the target list, clips the multimedia file currently being played to obtain clipped content. The embodiment of the present invention can match the content type in real time and efficiently recommend clipped motion content to the user. During video playback, a pre-acquired motion list is provided to the user, and the multimedia file is clipped according to the motion selected by the user. The clip is automatically generated in the background, thereby realizing a content self-production social system, improving the user's video consumption experience, and increasing user traffic and stickiness.

[0079] An embodiment of the present invention provides a computer-readable storage medium, which stores at least one executable instruction. When the executable instruction is executed on a computing device, the computing device executes the multimedia file clipping method in any of the above method embodiments.

[0080] The executable instructions may be used to cause the computing device to perform the following operations:

[0081] Determine, according to the content type of the multimedia file currently being played, a target list matching the content type;

[0082] In response to a user's selection operation on a target in the target list, the currently played multimedia file is clipped to obtain clipped content.

[0083] In an optional manner, determining a target list that matches the content type based on the content type of the multimedia file currently being played includes: matching the content type of the multimedia file currently being played in a type-target table in a database to obtain a target list that matches the content type; the type-target list in the database is a table of content type-corresponding targets obtained by pre-identifying targets in different types of multimedia file samples.

[0084] In an optional manner, the method of matching a type-target list in a database according to the content type of the multimedia file currently being played to obtain a target list matching the content type includes: matching a type-target list in a database according to the content type of the multimedia file currently being played to obtain each target corresponding to the multimedia file currently being played and the probabilities of the targets; and sorting the targets according to the probabilities of the targets to obtain a target list matching the content type.

[0085] In an optional manner, before determining a target list matching the content type based on the content type of the multimedia file currently being played, the method includes: acquiring the content type of a multimedia file sample; identifying the multimedia file sample to obtain a target corresponding to the multimedia file sample; and obtaining a type-target relationship based on the content type of the multimedia file sample and the target corresponding to the multimedia file sample.

[0086] In an optional manner, after identifying the multimedia file sample and obtaining the target corresponding to the multimedia file sample, the method includes: determining the number of times each target appears and the total number of times all targets appear in each type of multimedia file sample; and determining the probability of the relationship between the type and the target based on the number of times each target appears and the total number of times all targets appear.

[0087] In an optional manner, in response to the user's selection operation on a target in the target list, the currently playing multimedia file is clipped, and before obtaining the clipped content, it includes: displaying the target list matching the content type to the user; and receiving the user's selection of the target in the target list.

[0088] In an optional manner, in response to the user's selection operation on a target in the target list, the currently playing multimedia file is clipped to obtain clipped content, including: determining the position of the target selected by the user in the multimedia file; and clipping the currently playing multimedia file according to the position to obtain clipped content.

[0089] The working process of the multimedia file editing device of the embodiment of the present invention is substantially consistent with the method steps of the multimedia file editing method described above, and will not be repeated here.

[0090] The embodiment of the present invention determines a target list matching the content type according to the content type of the multimedia file currently being played, and in response to a user's selection operation on a target in the target list, clips the multimedia file currently being played to obtain clipped content. The embodiment of the present invention can match the content type in real time and efficiently recommend clipped motion content to the user. During video playback, a pre-acquired motion list is provided to the user, and the multimedia file is clipped according to the motion selected by the user. The clip is automatically generated in the background, thereby realizing a content self-production social system, improving the user's video consumption experience, and increasing user traffic and stickiness.

[0091] An embodiment of the present invention provides a multimedia file editing device, which is used to execute the multimedia file editing method.

[0092] An embodiment of the present invention provides a computer program, which can be called by a processor to enable a computing device to execute the multimedia file clipping method in any of the above method embodiments.

[0093] An embodiment of the present invention provides a computer program product, which includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions. When the program instructions are run on a computer, the computer executes the multimedia file clipping method in any of the above method embodiments.

[0094] The algorithm or display provided herein is not inherently related to any particular computer, virtual system or other equipment. Various general purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious to construct the structure required for this type of system. In addition, the embodiment of the present invention is not directed to any specific programming language yet. It should be understood that various programming languages ​​can be utilized to realize the content of the present invention described herein, and the description made to specific languages ​​above is for disclosing the best mode of the present invention.

[0095] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0096] Similarly, it should be understood that in order to streamline the present invention and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the embodiments of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this method of disclosure should not be interpreted as reflecting an intention that the claimed invention requires more features than those expressly recited in each claim.

[0097] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0098] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be understood as limitations on the order of execution.

Claims

1. A multimedia file editing method, characterized in that: The method comprises: Obtaining the content type of the multimedia file sample; identifying the multimedia file sample to obtain a target corresponding to the multimedia file sample; obtaining a type-target relationship based on the content type of the multimedia file sample and the target corresponding to the multimedia file sample; Wherein, the multimedia file sample is a video sample, and the target includes an action; the content type of the multimedia file sample is obtained; the multimedia file sample is identified to obtain the target corresponding to the multimedia file sample, and the type-target relationship is obtained according to the content type of the multimedia file sample and the target corresponding to the multimedia file sample, including: taking the existing video resources in the media resource library, randomly extracting N video samples according to a preset statistical sampling rule, and obtaining the content type of each video therein, wherein each video corresponds to at least one content type; identifying the video through an algorithm to obtain an action list for each video; associating multiple content types and actions of each video according to the Cartesian product, deduplicating the content type and action relationship in the N video samples, and then storing them in the content type and action table; wherein deduplication includes video deduplication and content type and action relationship deduplication; Determine the number of times each target appears in each type of multimedia file sample and the total number of times all targets appear; Determine the probability of the type-target relationship based on the number of times each target appears and the total number of times all targets appear; Determining a target list matching the content type according to the content type of the multimedia file currently being played, comprising: matching the content type of the multimedia file currently being played in a type-target table in a database to obtain a target list matching the content type; the type-target list in the database is a table of content type corresponding targets obtained by pre-identifying targets in different types of multimedia file samples; In response to a user's selection operation on a target in the target list, the currently played multimedia file is clipped to obtain clipped content.

2. The method according to claim 1, characterized in that The step of matching the content type of the multimedia file currently being played in the type-target list in the database to obtain the target list matching the content type includes: According to the content type of the multimedia file currently being played, matching is performed in the type-target list in the database to obtain each target corresponding to the multimedia file currently being played and the probability of the target; The targets are sorted according to their probabilities to obtain a target list matching the content type.

3. The method according to claim 1, characterized in that In response to the user's selection operation on the target in the target list, before clipping the currently played multimedia file to obtain the clipped content, the method includes: Displaying a target list matching the content type to the user; Receive a selection of a target in the target list by the user.

4. The method according to any one of claims 1 to 3, characterized in that: The step of clipping the currently played multimedia file to obtain clipped content in response to the user's selection operation on the target in the target list includes: determining a location of a user-selected target in the multimedia file; The currently played multimedia file is clipped according to the position to obtain clipped content.

5. A multimedia file editing device, characterized in that: The device comprises: A determination module is used to obtain the content type of a multimedia file sample; identify the multimedia file sample to obtain a target corresponding to the multimedia file sample; obtain a type-target relationship based on the content type of the multimedia file sample and the target corresponding to the multimedia file sample; wherein the multimedia file sample is a video sample, and the target includes an action; the obtaining of the content type of the multimedia file sample; identifying the multimedia file sample to obtain a target corresponding to the multimedia file sample, and obtaining a type-target relationship based on the content type of the multimedia file sample and the target corresponding to the multimedia file sample, including: taking existing video resources in a media resource library, randomly extracting N video samples according to a preset statistical sampling rule, and obtaining a content type of each video therein, wherein each video corresponds to at least one content type; identifying the video through an algorithm, and obtaining an action list of each video table; according to the Cartesian product, multiple content types and actions of each video are associated, and the content type and action relationship in N video samples are deduplicated, and then stored in the content type and action table; wherein deduplication includes video deduplication and content type and action relationship deduplication; determining the number of times each target appears and the total number of times all targets appear in each type of multimedia file sample; determining the probability of the type-target relationship according to the number of times each target appears and the total number of times all targets appear; determining the target list matching the content type according to the content type of the multimedia file currently being played, including: matching in the type-target table in the database according to the content type of the multimedia file currently being played, and obtaining the target list matching the content type; the type-target list in the database is a table of content type corresponding targets obtained by pre-identifying targets in different types of multimedia file samples; The clipping module is used to clip the currently played multimedia file to obtain clipped content in response to the user's selection operation on the target in the target list.

6. A computing device, characterized in that include: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the operation of the multimedia file editing method as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that: The storage medium stores at least one executable instruction, and when the executable instruction is executed on a computing device, the computing device executes the operation of the multimedia file editing method according to any one of claims 1 to 4.

Images