A video editing method, device, equipment and storage medium
By finding the original video data of similar business objects in the video data of business objects such as games and replacing the business information, the problem of long production cycles is solved, and efficient video data production and promotion are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANGHAI INGOT NETWORK TECH CO LTD
- Filing Date
- 2022-08-30
- Publication Date
- 2026-05-15
AI Technical Summary
In existing technologies, the video editing methods for updating and iterating business objects have a long production cycle for the video data of the business objects being promoted, which cannot meet the time-limited requirements of the shorter promotion cycle of business objects such as games.
By finding the original video data of a second business object that is similar to the first business object, replacing the business information in it, target video data suitable for promoting the first business object is generated.
It reduced the workload of artists, shortened the production cycle and costs, improved production efficiency, and met the promotion cycle requirements of business targets such as games.
Smart Images
Figure CN115410008B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of multimedia technology, and in particular to a video editing method, apparatus, device, and storage medium. Background Technology
[0002] In scenarios where games, electronic products, and other business entities are promoted, video data is often used to introduce these entities. Video data presents information about games, electronic products, and other business entities through sound and visuals, making the content intuitive, clear, and easy for users to read.
[0003] Currently, the speed of updating and iterating business objects is relatively fast, such as launching new business objects and updating the version of existing business objects. When updating and iterating business objects, new video data is usually configured for the business object for promotion and publicity.
[0004] If artists independently create video data adapted to each business object every time it is updated and iterated, this approach involves a large workload, a long production cycle, and will greatly increase costs in terms of manpower and time, resulting in low overall efficiency. Furthermore, for business objects such as games, the promotion cycle is generally short, and a long production cycle will not be able to meet the time-limit requirements. Summary of the Invention
[0005] This invention provides a video editing method, apparatus, device, and storage medium to address how to improve the efficiency of creating video data adapted to different channels when promoting business targets.
[0006] According to one aspect of the present invention, a video editing method is provided, comprising:
[0007] A first business object is identified, and the first business object is configured with multiple first business information items for promoting the first business object;
[0008] Find a second business object whose content is similar to the first business object;
[0009] The search content is the original video data that introduces the second business object, and the original video data contains multiple second business information for promoting the second business object;
[0010] In the original video data, multiple pieces of the second service information are sequentially replaced with multiple pieces of the first service information to obtain target video data whose content is an introduction to the first service object.
[0011] According to another aspect of the present invention, a video editing apparatus is provided, comprising:
[0012] A business object determination module is used to determine a first business object, which is configured with multiple first business information for promoting the first business object.
[0013] The business object search module is used to find a second business object whose content is similar to the first business object.
[0014] The original video data search module is used to search for original video data that introduces the second business object, wherein the original video data contains multiple second business information for promoting the second business object;
[0015] The target video data generation module is used to sequentially replace multiple pieces of the second business information with multiple pieces of the first business information in the original video data to obtain target video data whose content is an introduction to the first business object.
[0016] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:
[0017] At least one processor; and
[0018] A memory communicatively connected to the at least one processor; wherein,
[0019] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the video editing method according to any embodiment of the present invention.
[0020] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program configured to cause a processor to execute and implement the video editing method described in any embodiment of the present invention.
[0021] In this embodiment, a first business object is identified, which is configured with multiple first business information items for promoting the first business object. A second business object with similar content to the first business object is searched. The search content is original video data introducing the second business object, which contains multiple second business information items for promoting the second business object. In the original video data, the multiple second business information items are sequentially replaced with multiple first business information items to obtain target video data introducing the first business object. The similarity between the first and second business objects in content ensures that there is a certain degree of similarity between the first business object and the original video data. The original video data can be reused, and the second business information items in the original video data can be replaced with the first business information items, improving the matching degree between the target video data and the first business object. This can greatly reduce the workload of art staff, shorten the production cycle of the target video data, greatly reduce manpower and time costs, and improve overall efficiency. Furthermore, for business objects such as games, the shorter production cycle can meet the time limit requirements of their shorter promotion cycle.
[0022] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description
[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0024] Figure 1 This is a flowchart of a video editing method provided according to Embodiment 1 of the present invention;
[0025] Figure 2 This is a schematic diagram of editing target video data according to Embodiment 1 of the present invention;
[0026] Figure 3 This is an example diagram of banner information provided according to Embodiment 1 of the present invention;
[0027] Figure 4 This is an example diagram of an ending segment provided according to Embodiment 1 of the present invention;
[0028] Figure 5 This is a schematic diagram of the structure of a video editing device according to Embodiment 2 of the present invention;
[0029] Figure 6This is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention. Detailed Implementation
[0030] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0032] Example 1
[0033] Figure 1 This is a flowchart of a video editing method provided in Embodiment 1 of the present invention. This embodiment is applicable to situations where old business information in the video data of an old business object is replaced with new business information of a new business object, thereby editing the video data into video data of the new business object. This method can be executed by a video editing device, which can be implemented in hardware and / or software, and can be configured in an electronic device. Figure 1 As shown, the method includes:
[0034] Step 101: Determine the first business object.
[0035] In different business scenarios, there are objects that possess the business characteristics of that business scenario, referred to as business objects. Business objects can be physical items, such as mobile phones, tablets, smartwatches, etc., or they can be virtual items, mostly third-party applications, such as games, short video applications, live streaming applications, shopping applications, etc. This embodiment does not impose any restrictions on this.
[0036] To enable those skilled in the art to better understand the present invention, in this embodiment, a game is used as an example of a business object for illustration.
[0037] The types of games can include MOBA (Multiplayer Online Battle Arena), RPG (Role-playing game), SLG (Simulation Game), etc., and this embodiment does not limit them.
[0038] In this embodiment, a business object to be promoted is referred to as the first business object. The first business object can be a newly created business object or a business object obtained by updating the version, hardware, etc. of an old business object. This embodiment does not impose any restrictions on this.
[0039] Artists can create multiple business information entries for the primary business object to be promoted, thus obtaining the primary business information. The primary business information can be text, image data, audio data, etc. Its purpose is to promote the primary business object. Promotion can refer to the primary business information being used to inform users about the primary business object itself (such as a character or name in a game), or to attract users to purchase or download the primary business object (such as gift codes, coupons, etc.), or to indicate the method of purchasing or downloading the primary business object (such as the name and icon of an application distribution platform, the name and icon of a shopping platform, etc.).
[0040] So, when editing the content of the first business object as video data introducing the first business object, the first business object is configured with multiple first business information for promoting the first business object.
[0041] Step 102: Find a second business object whose content is similar to that of the first business object.
[0042] In this embodiment, the other business objects that have been promoted before, besides the first business object, are referred to as the second business object. When the second business object is promoted, video data (i.e., original video data) with the content of introducing the second business object has been produced. If the first business object and the second business object have certain similarities in content, then the original video data and the first business object will also have certain similarities in content, which can be used as the basis for editing the video data with the content of introducing the first business object.
[0043] Taking games as an example, a game developer first developed a game with a business simulation theme (i.e., the second business target). When promoting this game, they created video data (i.e., the original video data) that mainly told the story of running a business. The developer is now developing a card-based idle game with a business theme (i.e., the first business target) and is waiting to promote it. The two games are quite similar in theme, both being business management games. When designing the original video data, its storyline mainly leans towards the theme of the first game (i.e., business management) and does not showcase or explain much of the characteristics of the first game (such as characters, gameplay, etc.). This storyline is also applicable to the game to be promoted now, so it can be directly reused to create video data for promoting the current game (i.e., the target video data).
[0044] Therefore, this embodiment can match the first business object with each of the second business objects to find the second business object whose content is similar to that of the first business object.
[0045] When the art staff are familiar with the second business object, they can select the second business object whose content is similar to the first business object. However, when there are a large number of second business objects, the art staff may not be familiar with all of them. In order to improve the efficiency of selecting second business objects whose content is similar to the first business object, this embodiment can automatically select second business objects whose content is similar to the first business object. In order to further ensure the similarity of content, these second business objects can be provided to the art staff for review and confirmation.
[0046] In the specific implementation, on the one hand, a first vector representing the content features is extracted from the first business object; on the other hand, each second business object is queried in the local database, and a second vector representing the content features is extracted from the second business object. The similarity between the first business object and the second business object is calculated based on the first vector and the second vector, and then the second business object with similar content to the first business object is selected based on the similarity.
[0047] Generally, conditions such as similarity greater than or equal to a preset threshold, sequential sorting, and similarity ranking in the top n (n is a positive integer) can indicate that the first business object is similar to the second business object.
[0048] In one embodiment of the present invention, step 102 may include the following steps:
[0049] Step 1021: Identify multiple second business objects.
[0050] In this embodiment, multiple previously promoted second business objects can be recalled from the database, that is, the original video data of the second business object has been created to introduce the second business object.
[0051] Step 1022: Query the first character information, first type, and first art style information of the first business object, and query the second character information, second type, and second art style information of each second business object.
[0052] On the one hand, for a given primary business object, the primary character information, primary type, and primary art style information of the primary business object can be queried as the primary vector.
[0053] On the other hand, for each second business object, the second character information, second type, and second art style information of each second business object can be queried separately as a second vector.
[0054] The information on the first person and the information on the second person can both be image data, which can be obtained from the first business object and the second business object through facial recognition detection, or can be provided by the R&D personnel, etc.
[0055] The first and second art style information can be textual information describing the art style, or multidimensional features representing the art style extracted using neural networks, and so on.
[0056] Taking games as an example, when publishing a game on an application distribution platform, it is required to upload several representative frames of the game, its type, and a brief description. This information is backed up and stored locally. If the representative frame contains characters from the game, then the representative frame can be selected as character information (i.e., first character information and second character information). If the brief description contains keywords and / or tags describing the art style, then the keywords and / or tags can be selected as art style information (i.e., first art style information and second art style information).
[0057] Step 1023: Calculate the first similarity between the first character information and the second character information, the second similarity between the first type and the second type, and the third similarity between the first art style information and the second art style information.
[0058] In this embodiment, a first similarity is calculated between the first person information of the first business object and the second person information of each second business object; a second similarity is calculated between the first type of the first business object and the second type of each second business object; and a third similarity is calculated between the first art style information of the first business object and the second art style information of each second business object.
[0059] Because the forms of character information (i.e., first character information and second character information), type (i.e. first type and second type), and art style information (i.e. first art style information and second art style information) are different, the methods for calculating similarity (i.e. first similarity, second similarity, and third similarity) are also different.
[0060] For example, if the person information is in the form of image data, the first person information and the second person information can be compared using hash algorithms (such as mean hash algorithm aHash, difference hash algorithm dHash, perceptual hash algorithm pHash, etc.), mean squared error (MSE), structural similarity (SSIM), peak signal-to-noise ratio (PSNR), etc. to calculate the first similarity.
[0061] If the type is an identifier within a certain range (e.g., 1 represents MOBA, 2 represents RGP, etc.), then the first type and the second type can be compared to see if they are the same. If they are the same, the second similarity between the first type and the second type is 1; if they are different, the second similarity between the first type and the second type is 0.
[0062] If the art style information is in the form of text information or multi-dimensional features, the third similarity between the first and second art style information can be calculated using methods such as minimum edit distance, Euclidean distance, cosine distance, Jaccard similarity, and Hamming distance.
[0063] Step 1024: Filter second business objects that are similar to the first business object based on the first similarity, the second similarity and the third similarity.
[0064] In this embodiment, the similarity between the first business object and the second business object can be evaluated by comprehensively considering the first similarity, the second similarity and the third similarity, so as to select the second business object whose content is similar to the first business object based on the similarity.
[0065] For example, the first similarity, second similarity and third similarity can be linearly fused into the similarity between the first business object and the second business object, the level to which the first similarity, second similarity and third similarity belong can be identified respectively, and these levels can be jointly mapped into the similarity between the first business object and the second business object, and so on.
[0066] Of course, the above-described method for filtering second business objects similar to the first business object is merely an example. In implementing this embodiment, other methods for filtering second business objects similar to the first business object can be set according to actual circumstances, and this embodiment does not impose any limitations on this. Furthermore, besides the above-described method for filtering second business objects similar to the first business object, those skilled in the art can also adopt other methods for filtering second business objects similar to the first business object according to actual needs, and this embodiment does not impose any limitations on this either.
[0067] Step 103: The search content is the original video data introducing the second business object.
[0068] If original video data introducing the second business object was created when promoting the second business object, and this original video data is stored in a local database, then when editing video data introducing the first business object, the original video data introducing the second business object can be retrieved from this database.
[0069] In practical implementation, the content of the raw video data can be divided into two main forms: game content and real-life storyline. Game content can include introductions to the user's control of the game, introductions to the game by a spokesperson, or introductions by a spokesperson wearing in-game costumes. Storyline can be further divided into the following categories:
[0070] 1. Pseudo-food sharing
[0071] The original video data included some food-related materials to attract users' attention and incorporate gameplay elements of eating food while playing games.
[0072] 2. Themes closely related to users' daily lives
[0073] The content of the raw video data closely reflects users' current lifestyles, integrating games into various aspects of life, such as playing games while eating or buying snacks. The first half of this type of material primarily features dialogue between two people, while the second half includes segments showcasing the game integration.
[0074] 3. Exaggerated situational drama
[0075] The raw video data contains material from sitcoms, some of which features exaggerated storylines designed to attract users' attention.
[0076] Of course, the above-described raw video data is merely an example. When implementing this embodiment, other raw video data can be set according to actual circumstances, and this embodiment does not impose any limitations on this. Furthermore, in addition to the above-described raw video data, those skilled in the art can use other raw video data as needed, and this embodiment does not impose any limitations on this either.
[0077] Multiple business information items have already been created for the second business object, resulting in the second business information. The type of second business information can be text information, image data, audio data, etc. Its purpose is to promote the second business object. Promotion can refer to the second business information being used to inform users about the second business object itself (such as a character or name in a game), or to attract users to purchase or download the second business object (such as gift codes, coupons, etc.), or to indicate the method of purchasing or downloading the second business object (such as the name and icon of an application distribution platform, the name and icon of a shopping platform, etc.).
[0078] This business information has been written into the original video data, and therefore the original video data contains multiple pieces of second business information for promoting the second business target.
[0079] Step 104: In the original video data, replace multiple second business information items with multiple first business information items in sequence to obtain target video data whose content is an introduction to the first business object.
[0080] Generally, the first business information configured for the first business object is paired with the second business information configured for the second business object, meaning that the type of each first business information is the same as the type of each second business information.
[0081] Each second business information can be detected in the original video data. For each second business information, the second business information is replaced with the corresponding first business information. The edited original video data is recorded as the target video data whose content is to introduce the first business object.
[0082] Furthermore, when detecting various second service information in the original video data, since the content of the second service information is somewhat intertwined and all the second service information is present on the screen of the original video data, some second service information may interfere with other second service information. In order to reduce the impact of such interference, this embodiment can analyze and formulate the order in which each second service information is replaced with the corresponding first service information based on the degree of interference between the various second service information, and then replace multiple second service information with multiple corresponding first service information in sequence according to this order.
[0083] In one embodiment of the present invention, such as Figure 2 As shown, the first business information includes the first banner information 213, the first icon 214, and the first ending segment 212, and the second business information includes the second banner information 222, the second icon 221, and the second ending segment 223.
[0084] Both the first and second banner information belong to banner information, such as... Figure 3 As shown, a banner is generally a rectangular piece of information located at the top 301 and / or the bottom 302. It can record information about the business object (i.e., the first business object and the second business object) itself (such as the game screen, the characters in the game, the name (such as "XX game")) and information that attracts users to purchase or download the first business object (such as gift codes (such as "CL888")).
[0085] Both the first icon and the second icon belong to the category of icons (Logos). A logo is the mark of the business object (i.e., the first business object and the second business object) itself. It can be a textual logo (such as a name) or a graphic logo.
[0086] Both the first and second ending segments are ending segments (ECs), such as... Figure 4 As shown, EC records information about the business objects (i.e., the first business object and the second business object) themselves (such as the character and name in the game), and the way the business objects are purchased or downloaded (such as the icon of the application distribution platform, the name (such as "ABC AppStreet", "EFG Play") and icon of the application distribution platform, the name and icon of the shopping platform, etc.).
[0087] In this embodiment, step 104 may include the following steps:
[0088] Step 1041: Detect and delete the second ending segment in the original video data; if the second ending segment has been deleted, proceed to step 1042.
[0089] Since the second ending segment does not contain information about the second banner, if the second banner information is detected first and then the second ending segment is detected, there is a certain probability that image data from the second ending segment will be extracted during the detection of the second banner information. Because there is no static content, this will prevent the detection of the second banner information; in other words, the second ending segment will interfere with the detection of the second banner information.
[0090] Therefore, as Figure 2 As shown, in this embodiment, the second end segment 212 can be detected in each frame image data 211 of the original video data 210 based on the characteristics of the second end segment. When the second end segment 212 is detected, the second end segment 212 is deleted.
[0091] In one detection method, considering that EC is generally located at the end of the video data and its duration is usually within a certain range, a time window can be added to the original video data starting from the end point of the original video data. This time window has a certain duration, so that it can cover the last certain duration of the original video data. For example, the time window covers the last 6 seconds of the original video data, and so on.
[0092] Extract multiple frames of image data within the time window from the original video data. Starting with the last frame of image data, traverse and check each frame of image data in reverse order. Each traversal and check of image data can be recorded as one round of check. When traversing to a certain frame of image data, the frame of image data can be recorded as the current frame of image data. That is, the current frame of image data is initially the last frame of image data.
[0093] In each round of inspection, optical character recognition (OCR) is performed on the current frame image data to identify the first text information.
[0094] Considering that when transitioning from other content to EC, there is usually a clear transition screen (such as a completely black screen), it is necessary to determine whether the first text information contains a first keyword that represents the end of the segment, such as the name of the operating system suitable for installing the second business object, the name of the application distribution platform used to download the second business object, etc.
[0095] If not, it means that a transition scene has been detected, and the current frame image data belongs to the transition scene. It can be determined that the data located after the current frame image data in the original video data is the second end segment. Therefore, the second end segment is removed from the original video data. After removing the second end segment, the duration of the original video data is reduced.
[0096] If so, it means that EC features have been detected, and the current frame image data belongs to the content of the second end segment. Therefore, the image data that is ranked before the current frame image data is moved forward as the new current frame image data and enters the next round of inspection until the second end segment is identified.
[0097] Step 1042: Detect and delete the second banner information in the original video data; if the second banner information data has been deleted, proceed to step 1043.
[0098] If the second ending segment has been deleted from the original video data, the remaining information in the original video data will be the second banner information and the second icon. Since the second banner information and the second icon may contain the same information, such as the name of the second business object, if the second icon is detected first and then the second banner information is detected, the second banner information may be misjudged as the second icon. That is, the second banner information will interfere with the detection of the second icon.
[0099] Therefore, in this embodiment, as Figure 2 As shown, if the second end segment 212 has been deleted from the original video data, the second banner information 213 can be detected in the original video data 210 after the second end segment 212 has been deleted, based on the characteristics of the second banner information 213. When the second banner information 213 is detected, the second banner information 213 is deleted.
[0100] In one embodiment of the present invention, step 1042 may further include the following steps:
[0101] Step 10421: Detect multiple discontinuous image data frames in the original video data that are unrelated to the second ending segment.
[0102] In this embodiment, when detecting and deleting the second ending segment, some second ending segments may be missed, especially transition scenes (such as completely black scenes). In this case, in order to avoid interference caused by the missed second ending segments, multiple frames of image data can be detected in the original video data after the second ending segments have been deleted. These image data are not continuous in time, that is, other image data are interspersed between these image data, and these image data are selected from areas that can obviously avoid the second ending segments, so that these image data are unrelated to the second ending segments.
[0103] In one example, such as Figure 2 As shown, a first range can be extended sequentially (from the start point to the end point) from the starting point of the original video data 210 of the deleted second end segment 212 as the head 215. The first range can be counted in terms of duration, frame number, etc.
[0104] In the original video data 210 after the deletion of the second ending segment 212, a second range is divided from the head 215 as the middle 216. The second range can be counted in terms of duration, frame number, etc. The middle 216 is located after the head 215, which means that any time point of the head 215 is less than any time point of the middle 216.
[0105] To maximize the spacing between the head 215 and the middle 216, and between the middle 216 and the tail 217, the middle 216 covers the image data located at the median value, where the median value is half of the total number of frames of the original video data 210 after the second end segment 212 has been deleted.
[0106] Starting from the interference point after the middle 216, extend the third range in reverse order (from the end point to the start point) to form the tail 217. The third range can be counted in terms of duration, number of frames, etc. The tail is located after the middle, which means that any time point of the tail is less than any time point of the middle.
[0107] The interference point is a time point that is more than a preset distance away from the end point of the original video data 210 of the deleted second end segment 212. This distance can be counted in terms of duration, frame count, etc., and is an empirical value. There may be a missing part of the second end segment 212 between the interference point and the end point of the original video data 210 of the deleted second end segment 212, but there is no missing part of the second end segment 212 within the third range.
[0108] One frame of image data is extracted from the head 215, the middle 216, and the tail 217 respectively, according to the random or specified frame position, that is, one frame of image data is extracted from the head 215, such as the first frame of image data; one frame of image data is extracted from the middle 216, such as the image data located at the median value or the image data rounded to the median value; and one frame of image data is extracted from the tail 217, such as the 10th frame of image data from the end.
[0109] In this example, the original video data after the second ending segment has been deleted is divided into head, middle and tail. One frame of image data is extracted from each of the head, middle and tail. The interval between the head, middle and tail is maximized, so that the interval between the three frames of image data is maximized and the content difference between the three frames of image data is maximized. This avoids the situation where the content of the three frames of image data is similar when the interval between the three frames of image data is small, which affects the detection of the second banner information.
[0110] Step 10422: Take the absolute value of the difference in grayscale between any two frames of image data.
[0111] Step 10423: Mark the first area with an absolute value of 0 as the second banner information.
[0112] When there is a moving object in the original video data after the second ending segment has been deleted, there will be a difference in grayscale between frames. Therefore, in this embodiment, the absolute value of the difference in grayscale between any two frames of image data can be taken to represent the change in grayscale. The change in grayscale for stationary objects is 0, while the change in grayscale for moving objects (especially the outline of moving objects) is non-zero.
[0113] Since the information of the second banner is generally fixed, the first region with an absolute value of 0 is marked as the information of the second banner, thus ensuring low detection complexity, small computational load, adaptability to various dynamic environments, and good stability.
[0114] Step 10424: If the second banner information is located at the top and / or bottom of the image data, then extract the data from other areas outside the first area in the original video data to filter out the second banner information.
[0115] Normally, the second banner information is located at the top and / or bottom of the image data. In this embodiment, other areas in the image data other than the top and / or bottom can be set as a mask. The mask is nested on each frame of image data in the original video data after the second end segment has been deleted. The data located in the mask is extracted as the updated original video data, thereby filtering out the second banner information in the updated original video data. After filtering out the second banner information, the duration of the original video data does not change, but the height of the image is reduced.
[0116] Step 1043: Detect the second icon in the original video data and replace the second icon with the first icon; if the first icon has been replaced, proceed to step 1044.
[0117] like Figure 2 As shown, if the second banner information 213 has been deleted from the original video data 210, then the second icon 214 remains in the original video data 210. At this time, the second icon 214 can be detected in each frame of image data 211 in the original video data 214 according to the characteristics of the second icon 214. If the second icon 214 is detected, it can be directly replaced with the first icon 221.
[0118] In one embodiment of the present invention, step 1043 may further include the following steps:
[0119] Step 10431: Identify the second text information in each frame of the original video data.
[0120] In the original video data after the second ending segment and the second banner information have been deleted, light-written character recognition is performed on each frame of image data to obtain the second text information.
[0121] Considering that the second business object is deployed in different countries or regions, and to facilitate users in different countries or regions to browse the original video data, the original video data uses languages from different countries or regions. Therefore, in order to ensure the accuracy of recognizing the second text information, the language used by the second business object can be determined, and an optical character model adapted to the language can be loaded.
[0122] Among them, the optical character model is used to provide the operation of optical character recognition. Language adaptation can refer to using the text of the language to train the optical character model so that the optical character model can recognize the text of the language.
[0123] Each frame of image data from the original video data (after deleting the second ending segment and the second banner information) is input into the optical character model to perform optical character recognition, thereby obtaining the second text information.
[0124] Furthermore, considering that some second icons contain long text, and that the second text information is usually identified by rectangular text boxes, text boxes located on the same line can be merged to obtain the complete second text information.
[0125] Step 10432: Detect the position of the second text information in the image data.
[0126] The second text information is usually identified by a rectangular text box. The corner or center point of the text box can be used as a representative to calculate the position of the corner or center point in the image data.
[0127] Step 10433: Calculate the fourth similarity between the second text information and the second keyword configured for the second business object.
[0128] In this embodiment, a second keyword, such as the name of the second business object, can be pre-configured for the second business object.
[0129] The fourth similarity between the second text information and the second keyword is calculated using methods such as minimum edit distance and Hamming distance.
[0130] Step 10434: If the location is adjacent to the corner point of the image data and the similarity is greater than the preset threshold, then it is determined that the second region where the second text information is located contains the second icon.
[0131] Generally, in order to reduce the interference of the second icon on the original video data, the second icon is usually placed near the corner. If the position of the currently detected second text information is adjacent to the corner of the image data (i.e. the second text information is close to the corner of the image data) and the similarity is greater than the preset threshold (indicating that the second text information is similar to the second keyword representing the second business object), it means that the probability that the second text information belongs to the second icon is high. Therefore, it can be determined that the second area where the second text information is located contains the second icon.
[0132] The second area where the second text information is located can be the text box of the second text information itself, or it can be an area obtained by extending a certain distance in all directions (up, down, left, and right) based on the text box of the second text information. This embodiment does not limit this.
[0133] Step 10435: Write the first icon on the second area to cover the second icon.
[0134] In this embodiment, the first icon is written sequentially on each of the second areas, so that the first icon covers the second icon.
[0135] In practice, if the first icon and the second icon have the same size, the first icon can be written sequentially on each of the second areas, thus ensuring that the first icon covers the second icon.
[0136] If the first icon and the second icon are not the same size, the first size of the first icon (including the first width W and the first height H) and the second size of the second area (including the second width W and the second height H) can be identified respectively.
[0137] The first icon is scaled (e.g., equal height scaling and / or equal width scaling) so that the scaled first size is greater than or equal to the second size, that is, the first width is greater than or equal to the second width and the first height is greater than or equal to the second height.
[0138] Write the scaled-up first icon on the second area to cover the second icon.
[0139] Step 1044: Add the first banner information and the first ending segment to the original video data to obtain target video data that introduces the first business object.
[0140] like Figure 2 As shown, for the original video data 210 that has had the second ending segment 212 and the second banner information 213 deleted and the second icon 214 replaced with the first icon 221, the first banner information 222 and the first ending segment 223 can be added respectively. At this time, the content of the original video data is to introduce the first business object. For easy distinction, it can be recorded as target video data 220.
[0141] In a specific implementation, for the original video data in which the second ending segment and the second banner information have been deleted and the second icon has been replaced with the first icon, each frame of image data in the original video data can be traversed, and the first banner information can be added to the head and / or bottom of the image data.
[0142] Since the design specifications for banner information are relatively stable, the size of the first banner information is basically the same as that of the second banner information. Therefore, the height of the original video data after adding the first banner information increases, restoring the original height.
[0143] By splicing the first ending segment at the end of the original video data, the target video data is obtained, which introduces the first business object.
[0144] Because there may be fluctuations when deleting the second ending segment, and some of the second ending segment may be missed, the duration of the original video data after adding the first ending segment increases, but it is not necessarily the same as the original duration.
[0145] The target video data can be output to the art staff for review. The art staff can then perform post-processing on the target video data according to the editing situation. For example, they can blur the first icon to make it appear more natural when it is written on the screen, delete the missing second ending segment in the target video data, and stylize the target video data as a whole to increase the difference between it and the original video data, and so on.
[0146] Once the processing is complete, the target video data can be published on designated channels (such as news, short videos, novel reading, sports and health, etc.). When the client accesses the channel, the target video data will be pushed to the client for playback. When the user is interested in the primary business object, they can search for the primary business object through the information in the target video data, for example, searching for and downloading games from an application distribution platform, etc.
[0147] In this embodiment, a first business object is identified, which is configured with multiple first business information items for promoting the first business object. A second business object with similar content to the first business object is searched. The search content is original video data introducing the second business object, which contains multiple second business information items for promoting the second business object. In the original video data, the multiple second business information items are sequentially replaced with multiple first business information items to obtain target video data introducing the first business object. The similarity between the first and second business objects in content ensures that there is a certain degree of similarity between the first business object and the original video data. The original video data can be reused, and the second business information items in the original video data can be replaced with the first business information items, improving the matching degree between the target video data and the first business object. This can greatly reduce the workload of art staff, shorten the production cycle of the target video data, greatly reduce manpower and time costs, and improve overall efficiency. Furthermore, for business objects such as games, the shorter production cycle can meet the time limit requirements of their shorter promotion cycle.
[0148] Example 2
[0149] Figure 5 This is a schematic diagram of the structure of a video editing device provided in Embodiment 3 of the present invention. Figure 3 As shown, the device includes:
[0150] The business object determination module 501 is used to determine a first business object, which is configured with multiple first business information for promoting the first business object.
[0151] The business object lookup module 502 is used to find a second business object whose content is similar to the first business object;
[0152] The original video data search module 503 is used to search for original video data that introduces the second business object, wherein the original video data contains multiple second business information for promoting the second business object;
[0153] The target video data generation module 504 is used to sequentially replace multiple second service information with multiple first service information in the original video data to obtain target video data whose content is an introduction to the first service object.
[0154] In one embodiment of the present invention, the business object lookup module 502 includes:
[0155] The candidate determination module is used to determine multiple second business objects;
[0156] The business feature query module is used to query the first character information, first type, and first style information of the first business object, and to query the second character information, second type, and second style information of each second business object.
[0157] The feature similarity calculation module is used to calculate the first similarity between the first character information and the second character information, the second similarity between the first type and the second type, and the third similarity between the first art style information and the second art style information, respectively.
[0158] The similarity filtering module is used to filter second business objects that are similar to the first business object based on the first similarity, the second similarity, and the third similarity.
[0159] In one embodiment of the present invention, the first service information includes first banner information, a first icon, and a first end segment, and the second service information includes second banner information, a second icon, and a second end segment;
[0160] The target video data generation module 504 includes:
[0161] The end segment deletion module is used to detect and delete the second end segment in the original video data; if the second end segment has been deleted, the banner information deletion module is invoked.
[0162] The banner information deletion module is used to detect and delete the second banner information in the original video data; if the second banner information data has been deleted, the icon replacement module is invoked.
[0163] The icon replacement module is used to detect the second icon in the original video data and replace the second icon with the first icon; if the first icon has been replaced, the business information addition module is invoked.
[0164] The business information adding module is used to add the first banner information and the first ending segment to the original video data respectively, so as to obtain target video data with the content introducing the first business object.
[0165] In one embodiment of the present invention, the end-fragment deletion module includes:
[0166] The time window adding module is used to add a time window to the original video data, starting from the end point of the original video data;
[0167] The window data extraction module is used to extract multiple frames of image data located within the time window from the original video data;
[0168] The text information recognition module is used to recognize first text information in the image data of the current frame, wherein the image data of the current frame is initially the image data of the last frame;
[0169] The keyword judgment module is used to determine whether the first text information contains the first keyword that represents the end of the segment; if not, the deletion determination module is called; if yes, the forward move module is called.
[0170] The deletion determination module is used to determine that the data located after the image data in the current frame in the original video data is the second end segment, and to remove the second end segment from the original video data;
[0171] The forward module is used to move forward to the image data that is ranked one position before the image data in the current frame, and then return to call the text information recognition module.
[0172] In one embodiment of the present invention, the banner information deletion module includes:
[0173] A multi-frame data extraction module is used to detect multiple discontinuous image data frames in the original video data that are unrelated to the second ending segment;
[0174] The grayscale difference calculation module is used to take the absolute value of the difference in grayscale between any two frames of the image data.
[0175] The second banner information marking module is used to mark the first area with an absolute value of 0 as the second banner information;
[0176] A non-banner data cropping module is used to crop data from the original video data in areas other than the first area if the second banner information is located at the top and / or bottom of the image data, so as to filter out the second banner information.
[0177] In one embodiment of the present invention, the multi-frame data extraction module includes:
[0178] A header segmentation module is used to sequentially extend a first range from the starting point of the original video data as a header.
[0179] The middle segmentation module is used to divide a second range from the head onwards in the original video data as the middle segment.
[0180] The tail segmentation module is used to extend a third range in reverse order starting from the interference point after the middle section, as the tail. The interference point is a time point that is more than a preset distance away from the end point of the original video data.
[0181] Each extraction module is used to extract one frame of image data from the head, the middle and the tail respectively.
[0182] In one embodiment of the present invention, the icon replacement module includes:
[0183] The text information detection module is used to identify second text information in each frame of image data of the original video data;
[0184] The position detection module is used to detect the position of the second text information in the image data;
[0185] The keyword matching module is used to calculate the fourth similarity between the second text information and the second keyword configured for the second business object;
[0186] The icon determination module is used to determine that the second region where the second text information is located contains the second icon if the position is adjacent to the corner point of the image data and the fourth similarity is greater than a preset threshold.
[0187] An icon overlay module is used to write the first icon on the second area to cover the second icon.
[0188] In one embodiment of the present invention, the text information detection module includes:
[0189] The language determination module is used to determine the language used by the second business object;
[0190] An optical character model loading module is used to load an optical character model adapted to the language.
[0191] The optical character model invocation module is used to input each frame of the original video data into the optical character model to perform optical character recognition and obtain the second text information.
[0192] In one embodiment of the present invention, the icon overlay module includes:
[0193] The size recognition module is used to recognize the first size of the first icon and the second size of the second area, respectively.
[0194] An icon scaling module is used to scale the first icon so that the scaled first size is greater than or equal to the second size;
[0195] The scaling and overlay module is used to write the scaled first icon on the second area to cover the second icon.
[0196] In one embodiment of the present invention, the business information adding module includes:
[0197] A banner information adding module is used to traverse each frame of image data in the original video data and add the first banner information to the beginning and / or bottom of the image data;
[0198] The end segment addition module is used to splice the first end segment at the end point of the original video data to obtain target video data whose content is an introduction to the first business object.
[0199] The video editing device provided in the embodiments of the present invention can execute the video editing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the video editing method.
[0200] Example 3
[0201] Figure 6 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.
[0202] like Figure 6 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.
[0203] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0204] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as video editing methods.
[0205] In some embodiments, the video editing method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the video editing method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the video editing method by any other suitable means (e.g., by means of firmware).
[0206] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.
[0207] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.
[0208] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.
[0209] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).
[0210] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.
[0211] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.
[0212] Example 4
[0213] This invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the video editing method provided in any embodiment of this invention.
[0214] In implementing the computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages or a combination thereof. Programming languages include object-oriented programming languages such as Java, Smalltalk, and C++, as well as conventional procedural programming languages such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0215] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.
[0216] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.
Claims
1. A video editing method, characterized in that, include: A first business object is identified, and the first business object is configured with multiple first business information items for promoting the first business object; Find a second business object whose content is similar to the first business object; The search content is the original video data that introduces the second business object, and the original video data contains multiple second business information for promoting the second business object; In the original video data, multiple pieces of second service information are sequentially replaced with multiple pieces of first service information to obtain target video data whose content is an introduction to the first service object; The first business information includes a first banner, a first icon, and a first ending segment; the second business information includes a second banner, a second icon, and a second ending segment. In the original video data, multiple pieces of second service information are sequentially replaced with multiple pieces of first service information to obtain target video data whose content introduces the first service object, including: The second ending segment is detected and deleted from the original video data; If the second ending segment has been deleted, then the second banner information is detected and deleted from the original video data; If the second banner information data has been deleted, then the second icon is detected in the original video data and replaced with the first icon; If the first icon has been replaced, the first banner information and the first ending segment are added to the original video data respectively to obtain target video data that introduces the first business object.
2. The method according to claim 1, characterized in that, The second business object whose search content is similar to the first business object includes: Identify multiple secondary business objects; Query the first character information, first type, and first art style information of the first business object, and query the second character information, second type, and second art style information of each second business object; Calculate the first similarity between the first character information and the second character information, the second similarity between the first type and the second type, and the third similarity between the first art style information and the second art style information, respectively. Based on the first similarity, the second similarity, and the third similarity, a second business object similar to the first business object is selected.
3. The method according to claim 1, characterized in that, The step of detecting and deleting the second ending segment from the original video data includes: A time window is added to the original video data, starting from the end point of the original video data; Extract multiple frames of image data located within the time window from the original video data; First text information is identified in the image data of the current frame, and the image data of the current frame is initially the image data of the last frame. Determine whether the first text information contains a first keyword representing the end of the segment; If not, then the data located after the image data in the current frame in the original video data is determined to be the second end segment, and the second end segment is removed from the original video data; If so, move forward to the image data that is ranked one position before the image data in the current frame, and return to perform the step of identifying the first text information in the image data of the current frame.
4. The method according to claim 1, characterized in that, The step of detecting and deleting the second banner information from the original video data includes: Detect multiple discontinuous frames of image data in the original video data that are unrelated to the second ending segment; Take the absolute value of the difference in grayscale between any two frames of the image data; The first region with an absolute value of 0 is marked as the second banner information; If the second banner information is located at the top and / or bottom of the image data, data from other areas outside the first area in the original video data are extracted to filter out the second banner information.
5. The method according to claim 4, characterized in that, The step of detecting multiple discontinuous image data frames in the original video data that are unrelated to the second ending segment includes: A first range is sequentially extended from the starting point of the original video data, serving as the header; In the original video data, a second range is defined as the middle section, starting from the head. Starting from the interference point after the middle section, extend the third range in reverse order as the tail. The interference point is the time point that is more than a preset distance away from the end point of the original video data. One frame of image data is extracted from each of the head, the middle and the tail.
6. The method according to claim 1, characterized in that, The step of detecting the second icon in the original video data and replacing the second icon with the first icon includes: Identify second text information in each frame of the original video data; Detect the position of the second text information in the image data; Calculate the fourth similarity between the second text information and the second keyword configured for the second business object; If the location is adjacent to a corner point of the image data and the fourth similarity is greater than a preset threshold, then it is determined that the second region where the second text information is located contains the second icon; Write the first icon on the second area to cover the second icon.
7. The method according to claim 6, characterized in that, The step of identifying second text information in each frame of image data of the original video data includes: Determine the language used by the second business object; Load an optical character model adapted to the language; Each frame of the original video data is input into the optical character model to perform optical character recognition, thereby obtaining the second text information.
8. The method according to claim 6, characterized in that, The step of writing the first icon on the second area to cover the second icon includes: The first size of the first icon and the second size of the second region are identified respectively. The first icon is scaled so that the scaled first size is greater than or equal to the second size; Write the scaled-up first icon on the second area to cover the second icon.
9. The method according to claim 1, characterized in that, The step of adding the first banner information and the first ending segment to the original video data to obtain target video data whose content introduces the first business object includes: Iterate through each frame of image data in the original video data and add the first banner information to the beginning and / or bottom of the image data; By splicing the first ending segment at the end of the original video data, target video data is obtained, which introduces the first business object.
10. A video editing device, characterized in that, include: A business object determination module is used to determine a first business object, which is configured with multiple first business information for promoting the first business object. The business object search module is used to find a second business object whose content is similar to the first business object. The original video data search module is used to search for original video data that introduces the second business object, wherein the original video data contains multiple second business information for promoting the second business object; The target video data generation module is used to sequentially replace multiple second business information with multiple first business information in the original video data to obtain target video data whose content is an introduction to the first business object; The first business information includes a first banner, a first icon, and a first ending segment; the second business information includes a second banner, a second icon, and a second ending segment. The target video data generation module includes: The end segment deletion module is used to detect and delete the second end segment in the original video data; if the second end segment has been deleted, the banner information deletion module is invoked. The banner information deletion module is used to detect and delete the second banner information in the original video data; if the second banner information data has been deleted, the icon replacement module is invoked. The icon replacement module is used to detect the second icon in the original video data and replace the second icon with the first icon; if the first icon has been replaced, the business information addition module is invoked. The business information adding module is used to add the first banner information and the first ending segment to the original video data respectively, so as to obtain target video data with the content introducing the first business object.
11. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the video editing method according to any one of claims 1-9.
12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the video editing method of any one of claims 1-9.