Method, device, electronic device, and medium for determining a method for adding an object

Abstract

In an embodiment of the present application, a method, an apparatus, an electronic device, and a medium for determining a method for adding an object are disclosed. A specific embodiment of the method includes the steps of: acquiring a target video and a set of objects to be added to the target video; determining a period corresponding to each storyboard of the target video; determining an appearance period of the objects to be added in the set of objects to be added in the target video based on the period corresponding to each storyboard; and generating object addition instruction information based on the determined appearance period, the object addition instruction information including time display information for indicating the appearance period of the objects to be added in the set of objects to be added in the target video. In this embodiment, automatic selection of the timing at which the objects to be added to the target video are added is realized, thereby improving the degree of integration and consistency of the objects to be added with the target video.

Description

[Technical field]

[0001] TECHNICAL FIELD Embodiments of the present application relate to the field of computer technology, and in particular to a method, apparatus, electronic device, and medium for determining an additional manner of an object. [Background technology]

[0002] 2. Description of the Related Art With the rapid development of Internet technology, it has become common to enrich video content by adding corresponding service objects at appropriate positions in the video.

[0003] Related approaches typically determine time nodes of interest in a video either manually or by random assignment. Summary of the Invention

[0004] In the embodiments of the present application, a method, an apparatus, an electronic device, and a medium are provided for determining an addition manner of an object.

[0005] In an embodiment of the first aspect of the present application, a method for determining an object adding manner is provided, the method includes the steps of: obtaining a target video and a set of added objects to be added to the target video, determining a period corresponding to each storyboard of the target video, determining an appearance period of the added objects in the added object set in the target video according to the period corresponding to each storyboard, and generating object adding instruction information according to the determined appearance period, the object adding instruction information including time indication information for indicating the appearance period of the added objects in the added object set in the target video.

[0006] In an embodiment of a second aspect of the present application, an apparatus for determining an object adding manner is provided, the apparatus includes: an acquisition unit configured to acquire a target video and a set of added objects to be added to the target video, a first determination unit configured to determine a period corresponding to each storyboard of the target video, a second determination unit configured to determine an appearance period of the added object in the added object set in the target video based on the period corresponding to each storyboard, and a generation unit configured to generate object adding instruction information based on the determined appearance period, the object adding instruction information including time indication information for indicating an appearance period of the added object in the added object set in the target video.

[0007] In an embodiment of a third aspect of the present application, there is provided an electronic device, comprising one or more processors and a storage device having one or more programs stored therein, the one or more processors, when executed by the one or more processors, causing the one or more processors to implement a method as described in any implementation of the first aspect.

[0008] In an embodiment of a fourth aspect of the present application, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, realises a method as described in any of the implementations of the first aspect.

[0009] Further, in an embodiment of the fifth aspect of the present application, there is provided a computer program product, the computer program product including a computer program, the computer program being stored in a readable storage medium, one or more processors of an electronic device being capable of reading the computer program from the readable storage medium, and the one or more processors executing the computer program causing the electronic device to perform the method described in any of the implementations of the first aspect.

[0010] Further, in an embodiment of the sixth aspect of the present application, there is provided a computer program, the computer program being stored in a readable storage medium, one or more processors of an apparatus being capable of reading the computer program from the readable storage medium, and the one or more processors executing the computer program causing the apparatus to perform the method described in any of the implementations of the first aspect. Effect of the Invention

[0011] In the method, device, electronic device and medium for determining the manner of adding an object according to the embodiment of the present application, a period corresponding to each storyboard of the target video is first determined, then an appearance period of the added object in the target video is determined based on the determined period, and object addition instruction information is generated, including time presentation information for indicating the appearance period of the added object in the target video. This realizes automatic selection of the timing of adding the added object to the target video, and ensures that the appearance period of the added object in the target video does not cross storyboards as much as possible, thereby improving the integration and consistency of the added object with the target video and improving user acceptance. Particularly in the field of video advertising, it can obviously increase the conversion rate of sticker advertising. [Brief description of the drawings]

[0012] Other characteristics, objects and advantages of the present application will become more apparent on reading the detailed description of the non-limiting embodiments which refer to the following drawings, in which: [Figure 1] FIG. 1 is a diagram of an example of a system architecture in which an embodiment of the present application can be utilized. [Diagram 2] 1 is a flowchart of an embodiment of a method for determining an addition manner of an object according to the present application. [Diagram 3] FIG. 2 is a schematic diagram of an application scene of a method for determining an object addition manner according to an embodiment of the present application; [Figure 4] 4 is a flowchart of another embodiment of a method for determining an addition manner of an object according to the present application. [Diagram 5] FIG. 2 is a structural schematic diagram of an embodiment of an apparatus for determining an object addition manner according to the present application; [Figure 6] FIG. 1 is a structural schematic diagram of an electronic device suitable for implementing an embodiment of the present application. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] The present application will now be described in more detail with reference to the drawings and examples. It should be understood that the specific embodiments described herein are merely for the purpose of illustrating the relevant invention, and are not intended to be limiting of the present disclosure. For the sake of convenience, only parts related to the present disclosure are shown in the drawings.

[0014] In addition, the embodiments and features of the embodiments of the present application can be combined with each other as long as no contradiction occurs. MODE FOR CARRYING OUT THE PRESENTLY PREFERRED EMBODIMENTS The present application will now be described in detail with reference to the accompanying drawings.

[0015] In FIG. 1, an example of an architecture 100 is shown in which the present method for determining the manner of addition of an object or an apparatus for determining the manner of addition of an object can be utilized.

[0016] 1, system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. Network 104 is a medium that provides a communication link between terminal devices 101, 102, 103 and server 105. Network 104 may include various topologies, such as, for example, wired or wireless communication links, or optical cables.

[0017] The terminal devices 101, 102, 103 interact with a server 105 via a network 104 to receive or send messages, etc. Various communication client applications may be installed on the terminal devices 101, 102, 103, such as, for example, a web browser application, a shopping application, a search application, an instant messaging tool, an email client, social networking software, a video editing application, etc.

[0018] The terminal devices 101, 102, and 103 may be hardware or software. If they are hardware, the terminal devices 101, 102, and 103 may be various electronic devices having a display and supporting video editing, including, but not limited to, a smartphone, a tablet computer, a laptop computer, and a desktop computer. If they are software, the terminal devices 101, 102, and 103 may be installed in the aforementioned electronic devices. They may be realized as multiple software or software modules (e.g., software or software modules for providing distributed services), or as a single software or software module. There is no specific limitation here.

[0019] The server 105 may be a server that provides various services, for example, a background server that supports video editing software installed in the terminal devices 101, 102, and 103. The background server performs processing such as analysis on the received target video and a set of objects to be added to the target video, and can feed back the generated processing result (for example, object addition instruction information) to the terminal device.

[0020] In addition, the target video and the set of objects to be added to the target video may be directly stored locally in the server 105, and the server 105 may directly extract and process the locally stored target video and the set of objects to be added to the target video. In this case, the terminal devices 101, 102, 103 and the network 104 may not exist.

[0021] The server may be hardware or software. If it is hardware, the server may be realized as a distributed server cluster consisting of multiple servers, or as a single server. If it is software, the server may be realized as multiple software programs or software modules (e.g., software programs or software modules that provide distributed services), or as a single software program or software module. There are no specific limitations here.

[0022] It should be noted that the method for determining an object adding manner according to the embodiment of the present application is generally performed by the server 105, and accordingly, the device for determining an object adding manner is generally provided in the server 105. Optionally, under the condition of meeting the computational capability requirements, the method for determining an object adding manner according to the embodiment of the present application may be performed by the terminal devices 101, 102, 103, and accordingly, the device for determining an object adding manner may be provided in the terminal devices 101, 102, 103.

[0023] It should be understood that the number of terminal devices, networks and servers in Figure 1 is exemplary, and any number of terminal devices, networks and servers may be included according to the needs of implementation.

[0024] Further, referring to Fig. 2, a flow chart 200 of an embodiment of a method for determining an object addition manner according to the present application is shown. The method for determining an object addition manner includes the following steps:

[0025] In step 201, a target video and a set of objects to be added to the target video are obtained.

[0026] In this embodiment, an entity (e.g., server 105 in FIG. 1 ) performing the method for determining a manner of adding an object may obtain a target video and a set of added objects to be added to the target video via wired or wireless connection. The target video may be a video to which the added objects are to be added. The target video may be, for example, a movie, a drama series, a short video, an electronic album, etc. The added objects may include various objects that can be added to the target video, including, but not limited to, at least one of an image, a text box, and a sticker.

[0027] As one example, the execution subject may obtain a target video and a set of objects to be added to the target video that are stored locally in advance. As another example, the execution subject may obtain a target video and a set of objects to be added to the target video from an electronic device (e.g., the terminal device in FIG. 1) that communicates with and is connected to the execution subject. As yet another example, the execution subject may obtain a target video locally, and obtain a set of objects to be added to the target video from an electronic device (e.g., the terminal device in FIG. 1) that communicates with and is connected to the execution subject.

[0028] In step 202, a period of time corresponding to each storyboard in the target video is determined.

[0029] In this embodiment, the executing entity can determine the period corresponding to each storyboard of the target video acquired in step 201 in various ways. As an example, the executing entity can determine the period corresponding to each storyboard of the target video using various shot boundary detection techniques. The shot boundary detection techniques include, but are not limited to, at least one of a pixel comparison method, a color histogram comparison method, a block matching comparison method, and a dual threshold detection method. The period corresponding to the storyboard can indicate the start and end times corresponding to the shots included in the target video. As an example, the first shot is from 0 minutes 0 seconds to 0 minutes 11 seconds, the second shot is from 0 minutes 12 seconds to 0 minutes 26 seconds, and the third shot is from 0 minutes 27 seconds to 1 minute 3 seconds.

[0030] In step 203, the appearance durations of the objects to be added in the set of objects to be added in the target video are determined based on the durations corresponding to each storyboard.

[0031] In this embodiment, the executing entity can determine the appearance period of the target in the target set in the target video in various ways based on the period corresponding to each storyboard determined in step 202. As an example, the executing entity first obtains the number of targets to be added to each storyboard. For example, when the duration indicated from the period corresponding to the storyboard is 0 to 10 seconds, one target can be assigned, and when the duration indicated from the period corresponding to the storyboard is 11 to 30 seconds, two targets can be assigned. After that, the executing entity can determine which storyboard of the target video the target in the target set is to be added to. For example, the executing entity can assign the target in the target set to each storyboard of the target video randomly, or in a preset order. For example, the executing entity can assign the target 1 to the first storyboard, the targets 3 and 5 to the second storyboard, and the targets 2, 4, and 6 to the third storyboard. Then, the execution subject can determine the appearance period of the target video of the object to be added, which is assigned to each storyboard, in various ways. For example, the execution subject randomly selects a period that satisfies an appearance duration condition (e.g., 2 seconds or more) in the period corresponding to the first storyboard as the appearance period of the object to be added 1 (e.g., from 8 seconds to 10 seconds). For example, the execution subject determines the appearance period of the object to be added 1 to be the intermediate period (e.g., from 5 seconds to 7 seconds) of the period corresponding to the first storyboard.

[0032] In some alternative implementations of this embodiment, the actor may next determine the duration of appearance of the objects to be added in the target video in the set of objects to be added. In step 1, the total duration of the target video is determined.

[0033] In these implementations, the actor can determine the total duration of the target video in a variety of ways.

[0034] In step 2, the initial appearance duration of the target video of the target object in the target object set is determined based on the total duration of the target video.

[0035] In those realization forms, the execution subject can determine the initial appearance period of the target to be added in the target video in the added target set in various manners based on the total duration of the target video determined in step 1. As an example, the execution subject first determines the initial appearance period of the target to be added in the target video in the added target set so that the target to be added is distributed as evenly as possible in the total duration of the target video. For example, the total duration of the target video including three storyboards is 1 minute 3 seconds, and the execution subject can determine the initial appearance period of the target to be added in the added target set as 0 minutes 8 seconds to 0 minutes 9 seconds, 0 minutes 17 seconds to 0 minutes 18 seconds, 0 minutes 26 seconds to 0 minutes 27 seconds, 0 minutes 35 seconds to 0 minutes 36 seconds, 0 minutes 44 seconds to 0 minutes 45 seconds, and 0 minutes 53 seconds to 0 minutes 54 seconds.

[0036] In step 3, based on matching between the determined initial appearance period and the period corresponding to each storyboard in the target video, an object to be added that spans the storyboard is selected from the set of objects to be added to generate a set of target objects.

[0037] In these realization forms, the execution subject generates a target object set by selecting an object to be added that spans storyboards from the target object set based on matching between the determined initial appearance period and the period corresponding to each storyboard of the target video. Here, "spanning storyboards" is used to characterize that the start or end time of the period corresponding to the storyboard of the target video is included in the initial appearance period of the object to be added. As an example, since the second shot is from 0 minutes 12 seconds to 0 minutes 26 seconds and the third shot is from 0 minutes 27 seconds to 1 minute 3 seconds, the execution subject can add the object to be added that has an initial appearance period from 0 minutes 26 seconds to 0 minutes 27 seconds to the target object set.

[0038] In step 4, the appearance duration of the added object in the added object set in the target video is determined based on the adjustment to the initial appearance duration of the target object in the target object set.

[0039] In these implementations, the execution entity may select and adjust an initial appearance period of a target object from a target object set, and then the execution entity can determine an appearance period in the target video of the target object in the target object set based on the adjusted initial appearance period and the initial appearance period that does not require adjustment.

[0040] According to the optional implementation form, the execution entity first determines the initial appearance period of the object to be added, then adjusts according to the initial appearance period, and finally determines the appearance period of the object to be added in the target video in the object to be added set, which reduces the amount of calculation compared with directly determining the appearance period of each object to be added, and can flexibly adjust the generation method of the appearance period according to actual needs.

[0041] Optionally, in a next step, the execution entity can determine the duration of appearance of the target object in the target object set in the target video. In S1, it is determined whether a target object in a target object set satisfies a preset time adjustment condition.

[0042] In those implementations, the execution entity can determine whether the target objects in the target object set generated in step 3 satisfy the preset time adjustment condition in various ways.

[0043] Optionally, in a next step, the execution entity can determine whether the target objects in the target object set meet a preset time adjustment condition. In S11, a shot change time corresponding to a target object in a target object set is determined based on the determined initial appearance period and a period corresponding to each storyboard of the target video.

[0044] In those realizations, the execution entity can determine the shot change time corresponding to the target object in the target object set in various manners based on the determined initial appearance period and the period corresponding to each storyboard of the target video. The shot change time corresponding to the target object is for characterizing the start or end time of the period corresponding to the storyboard of the target video within the initial appearance period of the target object. As an example, the execution entity may determine that the shot change time corresponding to the target object whose initial appearance period is from 0 minutes 26 seconds (start) to 0 minutes 27 seconds (end) is 0 minutes 27 seconds (start), that is, determine the change time between the second shot and the third shot.

[0045] In S12, the adjustment direction in the time dimension of the initial appearance period of the target object to be adjusted is determined based on the determined shot change time.

[0046] In these realization forms, the execution subject can determine the adjustment direction in the time dimension of the initial appearance period of the adjusted target object in various ways based on the determined shot change time. The adjusted target object may be any target object in the target object set. The adjustment direction in the time dimension may include an advance adjustment and a delay adjustment. As an example, the advance adjustment is to adjust the initial appearance period from 0 minutes 26 seconds to 0 minutes 27 seconds to 0 minutes 20 seconds to 0 minutes 21 seconds. As another example, the delay adjustment is to adjust the initial appearance period from 0 minutes 26 seconds to 0 minutes 27 seconds to 0 minutes 30 seconds to 0 minutes 31 seconds.

[0047] In those implementations, the execution entity may determine the adjustment direction in the time dimension randomly or based on a preset rule. For example, the preset rule is to adjust to the middle time (e.g., 1 / 2 of the total duration) of the target video in the time dimension. For another example, the preset rule is to adjust to the direction of the storyboard that has a large overlap with the period corresponding to the storyboard that the initial appearance period of the adjusted target object spans.

[0048] In S13, it is determined whether or not the initial appearance period of the adjusted target object after being adjusted along the adjustment direction overlaps with the initial appearance period of the added object of the storyboard in the adjustment direction.

[0049] In these implementations, the execution entity can determine in various ways whether the initial appearance period of the adjusted target object after being adjusted along the adjustment direction determined in step S12 overlaps with the initial appearance period of the added object of the storyboard in the adjustment direction.

[0050] According to the selectable implementation form, the execution subject can determine whether the initial appearance period of the adjustment target object after being adjusted along the adjustment direction overlaps with the initial appearance period of the object to be added in the storyboard in the adjustment direction, based on whether the period adjustment condition is satisfied. The storyboard in the adjustment direction may be one storyboard or a preset number of continuous storyboards (for example, the initial appearance period of the object to be added spans N storyboards), and is not limited here.

[0051] Optionally, in a next step, the executing entity can determine whether an initial appearance period of the adjusted target object after being adjusted along the adjustment direction overlaps with an initial appearance period of an added object of the storyboard in the adjustment direction. In S131, the appearance interval between the added target and the adjusted target target, which is closest to the initial appearance period of the adjusted target target in the adjustment direction, is determined.

[0052] In these realization forms, the execution subject may first determine an appearance interval between the adjusted target object and the added object closest to the initial appearance period of the adjusted target object in the adjustment direction determined in step S12. The appearance interval may include a time difference between an end time of the initial appearance period of the front added object and a start time of the initial appearance period of the adjacent rear added object in the time dimension.

[0053] In S132, based on a comparison between the appearance interval and the duration indicated by the initial appearance period of the adjusted target object, it is determined whether or not the initial appearance period of the adjusted target object after being adjusted along the adjustment direction overlaps with the initial appearance period of an added object of the storyboard in the adjustment direction.

[0054] In those realization forms, as one example, in response to determining that the appearance interval determined in step S131 is greater than the duration indicated from the initial appearance period of the adjusted target object, the execution subject can determine that the initial appearance period of the adjusted target object after being adjusted along the adjustment direction does not overlap with the initial appearance period of the added object of the storyboard in the adjustment direction. As another example, in response to determining that the appearance interval determined in step S131 is less than or equal to the duration indicated from the initial appearance period of the adjusted target object, the execution subject can determine that the initial appearance period of the adjusted target object after being adjusted along the adjustment direction overlaps with the initial appearance period of the added object of the storyboard in the adjustment direction.

[0055] According to the selectable implementation form, the execution entity can use a comparison of the appearance interval of adjacent added objects with the appearance duration of the target object as a basis for determining whether the period adjustment condition is satisfied.

[0056] Optionally, in a next step, the executing entity can determine whether an initial appearance period of the adjusted target object after being adjusted along the adjustment direction overlaps with an initial appearance period of an added object of the storyboard in the adjustment direction. In S13'1, the total duration of the initial appearance period of the object to be added corresponding to the target storyboard is determined.

[0057] In those realizations, the execution subject may first determine a target storyboard. The target storyboard generally coincides with the adjustment direction and overlaps with the initial appearance period of the adjusted target object. As an example, as described above, when the adjustment direction is an advance adjustment in the time dimension, the execution subject may determine the second storyboard as the target storyboard. When the adjustment direction is a delay adjustment in the time dimension, the execution subject may determine the third storyboard as the target storyboard. After that, the execution subject may determine the total duration of the initial appearance period of the added object corresponding to the target storyboard in various ways. As an example, the execution subject may determine the total duration as the sum of the portion of the initial appearance period of the added object other than the target object that overlaps with the period corresponding to the target storyboard and the duration indicated by the initial appearance period of the target object.

[0058] In S13'2, based on the total duration and the duration of the period corresponding to the target storyboard, it is determined whether or not the initial appearance period of the adjusted target object after being adjusted along the adjustment direction overlaps with the initial appearance period of the added object of the storyboard in the adjustment direction.

[0059] In those realization forms, as an example, in response to determining that the total duration determined in step S13'1 is smaller than the duration of the period corresponding to the target storyboard, the execution entity can determine that an initial appearance period of the adjusted target object after being adjusted along the adjustment direction does not overlap with an initial appearance period of an added object of the storyboard in the adjustment direction. As another example, in response to determining that the total duration determined in step S13'1 is larger than or equal to the duration of the period corresponding to the target storyboard, the execution entity can determine that an initial appearance period of the adjusted target object after being adjusted along the adjustment direction overlaps with an initial appearance period of an added object of the storyboard in the adjustment direction.

[0060] According to the selectable implementation form, the executing entity can use a comparison between the total duration of the initial appearance period of the object to be added corresponding to the target storyboard and the duration of the period corresponding to the target storyboard as a basis for determining whether or not the period adjustment condition is met.

[0061] In S14, it is determined whether or not the target object in the target object set satisfies a preset period adjustment condition based on whether or not the initial appearance period of the adjusted target object after being adjusted along the adjustment direction overlaps with the initial appearance period of the added object of the storyboard in the adjustment direction.

[0062] In those realization forms, the execution entity can determine whether the target object in the target object set satisfies the preset period adjustment condition in various manners based on whether the initial appearance period of the adjusted target object after being adjusted along the adjustment direction overlaps with the initial appearance period of the added object of the storyboard in the adjustment direction. As an example, in response to determining that the initial appearance period of the adjusted target object after being adjusted along the adjustment direction overlaps with the initial appearance period of the added object of the storyboard in the adjustment direction, the execution entity can determine that the preset period adjustment condition is not satisfied.

[0063] Optionally, in response to determining that the initial appearance period of the adjusted target object after being adjusted along the adjustment direction does not overlap with the initial appearance period of an added object of the storyboard in the adjustment direction, the executing entity may determine that the preset period adjustment condition is satisfied.

[0064] Optionally, the execution entity can also determine in a next step whether the target objects in the target object set meet a preset time adjustment condition. In response to determining in S141 that the initial appearance period of the adjusted target object after being adjusted along the adjustment direction overlaps with the initial appearance period of the added object of the storyboard in the adjustment direction, it is determined whether or not the initial appearance period of the adjusted target object after being adjusted along the direction opposite to the adjustment direction overlaps with the initial appearance period of the added object of the storyboard in the direction opposite to the adjustment direction.

[0065] In those implementations, in response to determining that the initial appearance period of the adjusted target object after being adjusted along the adjustment direction overlaps with the initial appearance period of the added object of the storyboard in the adjustment direction, the executing entity can determine, in a manner similar to step S13, whether the initial appearance period of the adjusted target object after being adjusted along the opposite direction to the adjustment direction overlaps with the initial appearance period of the added object of the storyboard in the opposite direction to the adjustment direction. The adjustment direction may be, for example, a forward adjustment, and in that case, the opposite direction to the adjustment direction may be a backward adjustment.

[0066] In S142, in response to a determination that the initial appearance period of the adjusted target object after being adjusted in the direction opposite to the adjustment direction does not overlap with the initial appearance period of an added object in the storyboard in the direction opposite to the adjustment direction, it is determined that the preset period adjustment condition is satisfied.

[0067] In those realization forms, in response to determining that the initial appearance period of the adjusted target object after being adjusted along the opposite direction to the adjustment direction does not overlap with the initial appearance period of an added object in the storyboard that is in the opposite direction to the adjustment direction, the executing entity can determine that a preset period adjustment condition is satisfied.

[0068] According to the optional implementation, by attempting to make adjustments in two directions in the time dimension, it is ensured that the added object does not cross the storyboard as much as possible, and the display effect of the added object in the target video is improved.

[0069] In S2, the initial appearance period of the target object satisfying the period adjustment condition is adjusted so that the appearance period of the adjusted target object does not cross the storyboard.

[0070] In these implementations, the execution entity can adjust the initial appearance period of a target object that satisfies the period adjustment condition determined in step S1 so that the adjusted appearance period of the target object does not span the storyboard.

[0071] According to the selectable embodiment, the execution entity adjusts the appearance period of the target object that satisfies the period adjustment condition so that the appearance period of the adjusted target object does not cross the storyboard. Early appearance period can be adjusted, so as to improve the display effect in the target video of the added object.

[0072] Optionally, the execution subject can adjust the initial appearance period of the target object that satisfies the period adjustment condition to a new storyboard, so that the objects to be added in the new storyboard are evenly distributed. As an example, the execution subject first adjusts the initial appearance period of the target object that satisfies the period adjustment condition to a new storyboard that matches the adjustment direction (e.g., a storyboard before the straddling storyboard). Then, as the number of objects to be added included in the new storyboard increases, if the execution subject adjusts the appearance period of the objects to be added included in the new storyboard, the objects to be added are evenly distributed in the time dimension in the new storyboard. This improves the appearance effect of the objects to be added in the target video, which is also helpful in improving the quality of service (e.g., the conversion rate of displayed information).

[0073] In step 204, a target addition instruction is generated based on the determined occurrence period.

[0074] In this embodiment, the execution subject can generate object addition instruction information in various manners based on the appearance period determined in step 203. The object addition instruction information may include time presentation information for indicating the appearance period of the object to be added in the added object set in the target video. The time presentation information may include a start time and an end time of the appearance period.

[0075] Further, please refer to FIG. 3. FIG. 3 is a schematic diagram of an application scenario of the method for determining an object addition manner according to an embodiment of the present application. In the application scenario of FIG. 3, a user 301 Terminal 302 to send a sticker set 3031 including four stickers to a background server 304. The background server 304 obtains a target video 3032 from a database server 305. Then, the background server 304 can use a shot boundary detection technique to determine a period (e.g., 306 in FIG. 3) corresponding to each storyboard of the target video 3032. Then, the background server 304 can determine an appearance period (e.g., 307 in FIG. 3) of the sticker in the sticker set 3031 in the target video 3032 based on the determined period corresponding to each storyboard. Next, the background server 304 can further generate target addition instruction information 308. The target addition instruction information 308 includes, for example, a prompting text "Please add a sticker" corresponding to the appearance period. Optionally, the background server 304 combines the object addition instruction information 308 with the target video 3032 to generate a video 309 containing object addition instruction information and transmits it to the terminal 302, so that when the target video 3032 is played for the period indicated from 307 in FIG. 3, the terminal 302 displays the prompt message "Please add a sticker."

[0076] Currently, one of the prior arts is to manually determine the time node for adding a target in a video, which is inefficient in adding a target to a video and requires labor costs. Another of the prior arts is to add a service target to different time nodes of a video playback by random allocation, and the relevance to the video is not considered when determining the adding timing, which results in poor integration and consistency with the video. In the method according to the embodiment described in the present application, first, a period corresponding to each storyboard of the target video is determined, then, based on the determined period, the appearance period of the added target in the target video is determined, and further, an object addition instruction information including time presentation information for indicating the appearance period of the added target in the target video is generated. Automatic selection of the timing for adding the added target to the target video is realized, and it is ensured that the appearance period of the added target in the target video does not cross the storyboard as much as possible, so that the integration and consistency of the added target with the target video are improved, and the user acceptance is improved. Particularly in the field of video advertising, the conversion rate of sticker advertising can be greatly increased.

[0077] Further, reference is made to Fig. 4, which illustrates a flow 400 of another embodiment of a method for determining the manner of adding an object. The method flow 400 for determining the manner of adding an object includes the following steps. In step 401, a target video and a set of objects to be added to the target video are obtained.

[0078] In step 402, a period of time corresponding to each storyboard in the target video is determined.

[0079] In step 403, the appearance durations of the objects to be added in the set of objects to be added in the target video are determined based on the durations corresponding to each storyboard.

[0080] The steps 401, 402, and 403 are respectively the same as steps 201, 202, and 203 in the above embodiment, and the above explanations regarding steps 201, 202, and 203 can also be referred to for steps 401, 402, and 403, so detailed explanations are omitted here.

[0081] In step 404, video frames matching the determined occurrence period are extracted from the target video.

[0082] In this embodiment, the executing entity of the method for determining the target addition manner (e.g., the server 105 in FIG. 1) can extract video frames matching the determined appearance period from the target video in various manners. As an example, the executing entity can randomly select video frames as matching video frames from the video frame sequence corresponding to the appearance period determined as above. As another example, the executing entity can select video frames corresponding to the start time and end time indicated by the appearance period as matching video frames from the video frame sequence corresponding to the appearance period determined as above.

[0083] In step 405, for the extracted video frames, objects of a preset category are detected, and position information of the objects of the preset category is generated.

[0084] In this embodiment, the executing subject can detect objects of a preset category in the video frames extracted in step 404 using various object detection techniques, and generate position information of the objects of the preset category, thereby obtaining position information corresponding to each of the objects of the preset category in each video frame extracted.

[0085] In some alternative implementations of this embodiment, the pre-defined categories of objects may include face images and text images.

[0086] In step 406, the display area of ​​the object to be added corresponding to the set of objects to be added is obtained.

[0087] In this embodiment, the execution subject can obtain the display area of ​​the target corresponding to the target set from a local or communication-connected electronic device (e.g., the terminal device in FIG. 1). The display area may indicate the area of ​​the target in the video frame.

[0088] In some alternative implementation forms of this embodiment, the execution subject can obtain the display area of ​​the object to be added corresponding to the object to be added set in the next step. In step 1, the aspect ratio and minimum width of the object to be added are received from the user.

[0089] In these implementations, the execution subject can receive the aspect ratio and minimum width of each target to be added transmitted from the user side (eg, the terminal device in FIG. 1).

[0090] In step 2, the number of characters contained in the target to be added is obtained.

[0091] In these implementations, the execution entity can obtain the number of characters included in each added object, for example, 10 characters, 8 characters, etc. The characters include, but are not limited to, Chinese characters, English letters, etc.

[0092] In step 3, the area of ​​the characters to be added is generated based on the number of characters and a preset font size.

[0093] In these realization forms, the execution subject can generate the character area of ​​the object to be added based on the size of the area occupied by the preset font and the number of characters acquired in step 2. As one example, the execution subject determines the product of the size of the area occupied by the preset font and the number of characters acquired in step 2 as the character area of ​​the object to be added. As another example, the execution subject determines the number of lines required (e.g., 2) based on the number of characters acquired in step 2 (e.g., 8) and the number of characters displayed in each line (e.g., 5) that is preset. Next, the execution subject can determine the product of the number of characters displayed in each line, the number of lines required, and the size of the area occupied by the preset font, as the character area of ​​the object to be added.

[0094] In step 4, the display area of ​​the object to be added is generated based on the character area.

[0095] In these implementations, the executing entity can generate the display area of ​​the added object in various ways based on the character area generated in step 3. The aspect ratio of the display area generally matches the aspect ratio of the added object sent by the user side, and the width of the display area is greater than or equal to the minimum width.

[0096] According to the alternative implementation, a display area that meets user needs (eg, aspect ratio requirements, minimum width requirements) can be generated based on the content contained in the object to be added.

[0097] In step 407, a solid object occlusion metric is determined when the display area is at different positions in the matching video frame based on the overlap between the display area and the positions indicated by the generated position information of the pre-defined category of objects.

[0098] In this embodiment, the execution subject may first determine an overlap degree between the display area acquired in step 406 and a position indicated by the position information of the object of the preset category generated in step 405. The overlap degree may be, for example, the area of ​​the overlapped portion, or the ratio of the overlapped portion to the display area. Next, the execution subject may determine an entity occlusion metric when the display area is at a different position of the matching video frame based on the determined overlap degree. The entity occlusion metric is for indicating the degree of occlusion between an entity in a video frame and the display area.

[0099] In some alternative implementation forms of this embodiment, the executing entity can determine, in a next step, an entity occlusion metric when the display area is at different positions in the matching video frame from the degree of overlap between the display area and a position indicated from the position information of the generated preset category object based on the facial images and text images included in the preset category object. In step 1, a first convolution metric between the display area and a position indicated by the position information of a generated face image is determined.

[0100] In these realization forms, the execution entity can determine a first overlap metric between the display area and a position indicated by position information of the facial image to be generated. The first overlap metric is for indicating a degree of occlusion between the facial image in the video frame and the display area. As an example, the execution entity first determines an overlap degree between the area occupied by the facial image indicated by position information of each facial image included in the matching image frame and the display area. Next, the execution entity determines the sum of each overlap degree as the first overlap metric. As another example, the execution entity determines the number of facial images having an overlapping portion between the area occupied by the facial image indicated by position information of each facial image included in the matching image frame and the display area as the first overlap metric.

[0101] In step 2, a second convolution metric between the display area and the position indicated by the position information of the generated character image is determined.

[0102] In those implementations, the execution entity may determine the second convolution metric in a manner similar to step 1 above.

[0103] In step 3, based on the generated first and second overlay metrics, a solid occlusion metric is determined when the display area is at different positions in the matching video frames.

[0104] In these implementations, the execution entity can perform comprehensive processing on the generated first overlap degree and second overlap degree in various manners (e.g., obtaining a maximum value, a minimum value, a weighted average, etc.) to determine an entity occlusion metric when the display area is at different positions in the matching video frame.

[0105] According to the selectable implementation form, the execution entity can generate a substance occlusion metric based on the object to be added and the degree of overlap between the face image and the text image, thereby fine-tuning the evaluation of the degree of substance occlusion when the object to be added is in different positions and providing a data basis for determining the final appearance position.

[0106] In step 408, the location of the added object in the matching video frame is determined based on the solid occlusion metric.

[0107] In this embodiment, the execution subject can determine the appearance position of the added object in the matching video frame in various ways based on the solid occlusion metric determined in step 407. As an example, the execution subject selects a position with the smallest degree of solid occlusion based on the determined solid occlusion metric as the appearance position of the added object in the matching video frame. As another example, the execution subject first selects an appearance position with a degree of solid occlusion that meets a preset requirement as a quasi-appearance position based on the determined solid occlusion metric. Then, the execution subject can determine the appearance position of the added object in the matching video frame from the quasi-appearance position in various ways. For example, the execution subject randomly selects an appearance position from the quasi-appearance positions. As another example, the execution subject determines the appearance position of the added object in the matching video frame from the quasi-appearance positions that is closest to the center position of the video frame among the quasi-appearance positions.

[0108] In some alternative implementations of this embodiment, the actor can determine, in a next step, the appearance position of the added object in the matching video frame based on the solid occlusion metric. In step 1, the position deviation metric of the object to be added is obtained.

[0109] In those implementations, the execution entity can obtain the position bias metric of the added object in various ways. The position bias metric may include at least one of a user position bias metric, a position saliency metric of the added object, and an appearance position rule metric of the added object. The user position bias metric can indicate the degree of matching between different positions in the video frame to which the added object is matched and a preset user bias position. The position saliency metric can indicate the saliency of the added object when it is at different positions in the video frame to which it is matched, for example, when it is at the top left, bottom left, top right, or bottom right position of the video frame, the saliency is low. The appearance position rule metric can indicate the regularity of the appearance position in the video frame to which each of the added objects whose appearance order is adjacent is matched. The regularity can be, for example, symmetrical, or in a clockwise or counterclockwise order.

[0110] In step 2, a position evaluation metric is generated based on the solid occlusion metric and the position bias metric when the display area is at different positions in the matching video frame.

[0111] In those implementations, the actor can generate a position evaluation metric when the display area is at different positions of the matching video frame in various manners based on the solid occlusion metric determined in step 407 and the position bias metric obtained in step 1. The value of the position evaluation metric may generally be negatively correlated to the degree of occlusion indicated by the solid occlusion metric. The value of the position evaluation metric may generally be positively correlated to at least one of the degree of matching, saliency, and regularity indicated by the position bias metric.

[0112] In step 3, the position corresponding to the maximum value of the position evaluation metric is determined as the appearance position of the object to be added in the matching video frame.

[0113] In these realization forms, the execution subject may determine the position with the highest position evaluation metric score generated in step 2 as the appearance position of the added object in the matching video frame. Optionally, when the value of the position evaluation metric is positively correlated with the degree of occlusion indicated by the solid occlusion metric or negatively correlated with at least one of the degree of matching, saliency and regularity indicated by the position bias metric, the execution subject may determine the position corresponding to the minimum value of the position evaluation metric as the appearance position of the added object in the matching video frame, and detailed description is omitted here.

[0114] In step 409, target addition instruction information is generated based on the determined appearance period and appearance position.

[0115] In this embodiment, the execution subject can generate target addition instruction information in various manners according to the appearance period determined in step 403 and the appearance position determined in step 408. The target addition instruction information may further include position instruction information for indicating the appearance position of the target to be added in the added target set in the video frame of the target video.

[0116] In some alternative implementations of this embodiment, the position indication information may include center coordinates for indicating a display position of the target to be added. The position indication information may include a dialog box to be displayed at the determined appearance position.

[0117] As can be seen from Fig. 4, the method flow 400 for determining the object adding manner of this embodiment shows the steps of determining the substance occlusion metric of the added object when it is at different positions in the video frame of the display area, and determining the appearance position in the video frame according to the determined substance occlusion metric. Thus, the object adding indication information generated in the technical solution described in this embodiment may indicate the appearance period or the appearance position, thereby realizing the automatic determination of the appearance position and appearance period of the added object in the target video, effectively reducing the cost of manual addition and improving efficiency.

[0118] Further, please refer to Fig. 5. In the present application, as a realization of the method shown in each of the figures, an embodiment of an apparatus for determining an additional manner of an object is provided. The embodiment of the apparatus corresponds to the embodiment of the method shown in Fig. 2 or Fig. 4, and in particular, the apparatus can be used in various electronic devices.

[0119] As shown in FIG. 5, the apparatus 500 for determining an object adding manner according to the present embodiment includes an acquisition unit 501, a first determination unit 502, a second determination unit 503, and a generation unit 504. The acquisition unit 501 is configured to acquire a target video and a set of added objects to be added to the target video. The first determination unit 502 is configured to determine a period corresponding to each storyboard of the target video. The second determination unit 503 is configured to determine an appearance period of the added object in the added object set in the target video according to the period corresponding to each storyboard. The generation unit 504 is configured to generate object adding instruction information according to the determined appearance period, where the object adding instruction information includes time presentation information for indicating an appearance period of the added object in the added object set in the target video.

[0120] In this embodiment, in the apparatus 500 for determining an object addition method, the specific processing and technical effects brought about by the acquisition unit 501, the first determination unit 502, the second determination unit 503 and the generation unit 504 can refer to the relevant descriptions of steps 201, 202, 203 and 204 of the embodiment corresponding to FIG. 2, respectively, so detailed descriptions are omitted here.

[0121] In some alternative implementations of this embodiment, the second determination unit 503 may include a first determination subunit (not shown), a second determination subunit (not shown), a selection subunit (not shown), and an adjustment subunit (not shown). The first determination subunit may be configured to determine a total duration of the target video. The second determination subunit may be configured to determine an initial appearance period in the target video of the target object in the target object set based on the total duration of the target video. The selection subunit may be configured to select a target object spanning a storyboard from the target object set based on matching between the determined initial appearance period and a period corresponding to each storyboard in the target video to generate a target object set. The adjustment subunit may be configured to determine an appearance period in the target video of the target object in the target object set based on an adjustment to the initial appearance period of the target object in the target object set.

[0122] In some alternative implementations of this embodiment, the adjustment subunit may include a first determination module (not shown) and an adjustment module (not shown). The first determination module may be configured to determine whether a target object in a target object set satisfies a preset period adjustment condition. The adjustment module may be configured to adjust an initial appearance period of a target object that satisfies the period adjustment condition, such that the appearance period of the adjusted target object does not cross a storyboard.

[0123] In some alternative implementations of this embodiment, the first determination module may include a first determination submodule (not shown), a second determination submodule (not shown), a third determination submodule (not shown), and a fourth determination submodule (not shown). The first determination submodule may be configured to determine a shot change time corresponding to a target object in a target object set based on the determined initial appearance period and a period corresponding to each storyboard of the target video. The second determination submodule may be configured to determine an adjustment direction in a time dimension of the initial appearance period of the adjusted target object based on the determined shot change time. The third determination submodule may be configured to determine whether the initial appearance period of the adjusted target object after being adjusted along the adjustment direction and the initial appearance period of the added object of the storyboard in the adjustment direction overlap. The fourth determination submodule may be configured to determine whether the target object in the target object set meets a preset period adjustment condition based on whether the initial appearance period of the adjusted target object after being adjusted along the adjustment direction and the initial appearance period of the added object of the storyboard in the adjustment direction overlap.

[0124] In some alternative implementations of this embodiment, the third determination submodule may be further configured to: determine an appearance interval between the added object and the adjusted target object that is closest to the initial appearance period of the adjusted target object in the adjustment direction; and determine whether the initial appearance period of the adjusted target object after being adjusted along the adjustment direction overlaps with the initial appearance period of the added object of the storyboard in the adjustment direction based on a comparison between the appearance interval and a duration indicated by the initial appearance period of the adjusted target object.

[0125] In some alternative implementations of this embodiment, the third determining submodule may be further configured to: determine a total duration of an initial appearance period of an added object corresponding to a target storyboard, where the target storyboard is consistent with the adjustment direction and overlaps with the initial appearance period of the adjusted target object; and determine whether the initial appearance period of the adjusted target object after being adjusted along the adjustment direction overlaps with the initial appearance period of the added object of the storyboard in the adjustment direction based on the total duration and the duration of the period corresponding to the target storyboard.

[0126] In some alternative implementations of this embodiment, the fourth determination submodule may be further configured to: determine that a preset period adjustment condition is satisfied in response to determining that an initial appearance period of the adjusted target object after being adjusted along the adjustment direction does not overlap with an initial appearance period of an added object of the storyboard in the adjustment direction.

[0127] In some alternative implementations of this embodiment, the fourth determination submodule may be further configured as follows: in response to determining that the initial appearance period of the adjusted target object after being adjusted along the adjustment direction overlaps with the initial appearance period of the added object of the storyboard in the adjustment direction, determine whether the initial appearance period of the adjusted target object after being adjusted along the opposite direction to the adjustment direction overlaps with the initial appearance period of the added object of the storyboard in the opposite direction to the adjustment direction; in response to determining that the initial appearance period of the adjusted target object after being adjusted along the opposite direction to the adjustment direction does not overlap with the initial appearance period of the added object of the storyboard in the opposite direction to the adjustment direction, determine that a preset period adjustment condition is satisfied.

[0128] In some alternative implementations of this embodiment, the adjustment module may be further configured to: adjust initial appearance periods of target objects that satisfy a period adjustment condition in the new storyboard, so that the objects to be added in the new storyboard are uniformly distributed.

[0129] In some alternative implementations of this embodiment, the object addition instruction information may further include position instruction information for indicating an appearance position of the object to be added in the set of objects to be added in the video frame of the target video. The generating unit 504 may include an extracting subunit (not shown), a first generating subunit (not shown), an acquiring subunit (not shown), a third determining subunit (not shown), a fourth determining subunit (not shown), and a second generating subunit (not shown). The extracting subunit may be configured to extract a video frame matching the determined appearance period from the target video. The first generating subunit may be configured to detect an object of a preset category for the extracted video frame and generate position information of the object of the preset category. The acquiring subunit may be configured to acquire a display area of ​​the object to be added corresponding to the set of objects to be added. The third determining subunit may be configured to determine a substantial object occlusion metric when the display area is at a different position of the matching video frame based on the overlapping degree between the display area and the position indicated by the generated position information of the object of the preset category. The fourth determining subunit may be configured to determine an appearance position of the object to be added in the matching video frame based on the solid occlusion metric. The second generating subunit may be configured to generate object adding instruction information based on the determined appearance duration and appearance position.

[0130] In some alternative implementations of this embodiment, the predefined category of objects may include face images and text images. The third determination subunit may include a fifth determination submodule (not shown), a sixth determination submodule (not shown), and a seventh determination submodule (not shown). The fifth determination submodule may be configured to determine a first overlay metric between the display area and a position indicated by position information of the generated face image. The sixth determination submodule may be configured to determine a second overlay metric between the display area and a position indicated by position information of the generated character image. The seventh determination submodule may be configured to determine an entity occlusion metric when the display area is at a different position of the matching video frame based on the generated first overlay metric and second overlay metric.

[0131] In some alternative implementations of this embodiment, the acquisition subunit may include a receiving module (not shown), a first acquiring module (not shown), a first generating module (not shown), and a second generating module (not shown). The receiving module may be configured to receive an aspect ratio and a minimum width of the object to be added sent from a user side. The first acquiring module may be configured to acquire the number of characters included in the object to be added. The first generating module may be configured to generate a character area of ​​the object to be added based on the number of characters and a preset font size. The second generating module may be configured to generate a display area of ​​the object to be added based on the character area. The aspect ratio of the display area may match the aspect ratio of the object to be added sent from a user side. The width of the display area is greater than or equal to the minimum width.

[0132] In some alternative implementations of this embodiment, the fourth determination subunit may include a second acquisition module (not shown), a third generation module (not shown), and a second determination module (not shown). The second acquisition module may be configured to acquire a position bias metric of the object to be added. The position bias metric may include at least one of a user position bias metric, a position saliency metric of the object to be added, and an appearance position rule metric of the object to be added. The third generation module may be configured to generate a position evaluation metric when the display area is at a different position in the matching video frame based on the solid occlusion metric and the position bias metric. The second determination module may be configured to determine the position corresponding to the maximum value of the position evaluation metric as the appearance position of the object to be added in the matching video frame.

[0133] In the device according to the embodiment described in the present application, first, the first determining unit 502 determines the period corresponding to each storyboard of the target video, then the second determining unit 503 determines the appearance period of the added target in the target video based on the determined period, and further, the generating unit 504 generates the target addition instruction information including time presentation information for indicating the appearance period of the added target in the target video. The timing of adding the added target to the target video is automatically selected, and the appearance period of the added target in the target video is guaranteed not to cross the storyboard as much as possible, so that the integration degree and consistency of the added target with the target video are improved, and the user acceptance is improved. Particularly in the field of video advertising, the conversion rate of sticker advertising can be greatly increased.

[0134] Reference is now made to FIG. 6, which illustrates a structural schematic diagram of an electronic device (e.g., the server or terminal device of FIG. 1) 600 suitable for implementing the embodiments of the present application. The terminal devices according to the embodiments of the present application include, but are not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, personal digital assistants (PDAs), tablet personal computers (PADs), portable multimedia players (PMPs), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital televisions and desktop computers. The terminal devices or servers illustrated in FIG. 6 are merely exemplary and are not intended to limit the functions and scope of use of the embodiments of the present application.

[0135] 6, the electronic device 600 may include a processing unit (e.g., a central processing unit, a graphic processor, etc.) 601 that can perform various suitable operations and processes based on programs stored in a read-only memory (ROM) 602 or loaded from a storage device 608 into a random access memory (RAM) 603. The RAM 603 also stores various programs and data necessary for the operation of the electronic device 600. The processing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0136] In general, devices that can be connected to the I / O interface 605 include input devices 606, such as a touch panel, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 607, such as a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 608, such as a magnetic tape, hard disk, etc.; and communication devices 609. The electronic device 600 can exchange data with other devices through wireless or wired communication devices 609. Although FIG. 6 shows the electronic device 600 having various devices, it should be understood that the electronic device 600 does not necessarily implement or include all of the devices shown. Instead, the electronic device 600 may implement or include more or fewer devices. Each block shown in FIG. 6 may represent one device or may represent multiple devices in some cases.

[0137] In particular, according to an embodiment of the present application, the process described above with reference to the flowcharts may be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product having a computer program carried on a computer-readable medium, the computer program including a program code for performing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network by the communication device 609, or may be installed from the storage device 608, or may be installed from the ROM 602. When the computer program is executed by the processing device 601, the steps of the method of the embodiment of the present application are performed, and the functions defined above are realized.

[0138] In an embodiment of the present disclosure, a computer program is further provided, the computer program being stored in a readable storage medium, one or more processors of an electronic device can read the computer program from the readable storage medium, and the one or more processors execute the computer program, causing the electronic device to implement the technical solution according to any of the embodiments.

[0139] It should be noted that the computer-readable medium described in the embodiments of the present application may be a computer-readable signal medium or a computer-readable storage medium, or any combination of both. The computer-readable storage medium may be, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples of the computer-readable storage medium include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM, electrically programmable read-only memory, or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM, compact disc read-only memory), an optical storage component, a magnetic storage component, or any suitable combination thereof. In the embodiments of the present application, the computer-readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in combination with an instruction execution system, device, or component. In the present embodiment, the computer-readable signal medium may include a propagated data signal in baseband or as part of a carrier wave on which the computer-readable program code is borne. Such a propagated data signal may be in various forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination thereof. The computer-readable signal medium may be any computer-readable medium other than a computer-readable storage medium, which may transmit, propagate, or transport a program for use by or in combination with an instruction execution system, device, or component.The program code contained in the computer readable medium may be transmitted over any suitable medium, including, but not limited to, electrical wire, optical cable, radio frequency (RF), or the like, or any suitable combination thereof.

[0140] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated in the electronic device. The computer-readable medium bears one or more programs, and when the one or more programs are executed by the electronic device, the electronic device obtains a target video and a set of added objects to be added to the target video, determines a period corresponding to each storyboard of the target video, determines an appearance period of the added objects in the added object set in the target video based on the period corresponding to each storyboard, and generates object addition instruction information based on the determined appearance period, where the object addition instruction information includes time presentation information for indicating the appearance period of the added objects in the added object set in the target video.

[0141] Computer program code for carrying out the operations of the embodiments of the present application can be written in one or more programming languages ​​or combinations thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and even conventional procedural programming languages ​​such as "C", Python, or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a single independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. When a remote computer is involved, the remote computer may be connected to the user's computer by any type of network, including a local area network (LAN), a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider).

[0142] The flowcharts and block diagrams in the drawings illustrate the architecture, functions, and operations that can be implemented in the systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flowcharts or block diagrams may represent a module, program segment, or part of code, which includes one or more executable instructions for implementing a certain logical function. It should be noted that in some alternative implementations, the functions shown in the blocks may be executed in a different order than that shown in the drawings. For example, two blocks shown as connected may actually be executed substantially in parallel, or may sometimes be executed in the reverse order, as determined by the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented in a dedicated system consisting of hardware that executes a certain function or operation, or may be implemented in a combination of dedicated hardware and computer instructions.

[0143] The units mentioned in the description of the embodiments of the present application may be implemented as software or hardware. The described units may be provided in a processor. For example, the processor may be described as including an acquisition unit, a first determination unit, a second determination unit, and a generation unit. The names of the units may not constitute limitations on the units themselves in some cases, for example, the acquisition unit may be described as "a unit for acquiring a target video and an added object set to be added to the target video."

[0144] The above description is merely a description of the preferred embodiment of the present application and the technical principles used. Those skilled in the art should understand that the scope of the disclosure of the embodiments of the present application is not limited to the technical solution of the specific combination of the above technical features, but also includes other technical solutions obtained by arbitrarily combining the above technical features or equivalent features without departing from the spirit of the disclosure. For example, a technical solution obtained by replacing the above features with (but not limited to) technical features having similar functions as those disclosed in the embodiments of the present application.

[0145] [CROSS REFERENCE TO RELATED APPLICATIONS] This application claims priority to a Chinese patent application filed with the State Intellectual Property Office of the People's Republic of China on August 26, 2020, bearing application number 202010870279.4 and entitled "Method, Apparatus, Electronic Device and Medium for Determining the Addition Method of an Object," which is incorporated herein by reference in its entirety.

Claims

1. obtaining a target video and a set of objects to be added to the target video; determining a duration corresponding to each storyboard of the target video; determining an appearance duration of the objects to be added in the set of objects to be added in the target video based on a duration corresponding to each of the storyboards; generating target addition instruction information based on the determined appearance period, the target addition instruction information including time presentation information for indicating an appearance period of the target to be added in the target set of targets to be added in the target video; The step of determining an appearance period of the target video of the objects to be added in the set of objects to be added based on a period corresponding to each of the storyboards includes: determining a total duration of the target video; determining an initial appearance period in the target video of a target object in the set of targets to be added based on a total duration of the target video; A step of selecting an object to be added across storyboards from the set of objects to be added based on matching between the determined initial appearance period and a period corresponding to each storyboard of the target video, thereby generating a target set of objects to be added that are adjustment targets; The step of setting an initial appearance period of the target object not spanning a storyboard as an appearance period of the target object not spanning the storyboard in the target video, adjusting an initial appearance period of the target object in the target set of targets to be added so as not to span the storyboard of the target video, and setting the initial appearance period after the adjustment as an appearance period of the target object to be added in the target video, A method for determining a target addition scheme implemented by an electronic device.

2. The step of determining the appearance period of the target object in the target object set in the target video based on an adjustment to an initial appearance period of the target object in the target object set includes: determining whether the target object in the target object set satisfies a preset time adjustment condition; The method according to claim 1 , further comprising a step of adjusting an initial appearance period of a target object to be added that satisfies the period adjustment condition so that the appearance period of the target object to be added after the adjustment does not span a storyboard.

3. The step of determining whether the target object to be added in the target object set satisfies a preset period adjustment condition includes: determining a shot change time corresponding to a target object in the set of target objects to be added based on the determined initial appearance period and a period corresponding to each storyboard of the target video; A step of determining an adjustment direction in a time dimension of an initial appearance period of a target object to be added based on the determined shot change time; determining whether an initial appearance period of the target added object after being adjusted along the adjustment direction overlaps with an initial appearance period of one or more added objects of the storyboard in the adjustment direction; and determining whether a target object to be added in the set of target objects to be added satisfies a preset period adjustment condition based on whether an initial appearance period of the target object to be added after being adjusted along the adjustment direction overlaps with an initial appearance period of one or more objects to be added in the storyboard in the adjustment direction; The step of determining whether a target object to be added in the set of target objects to be added satisfies a preset period adjustment condition based on whether an initial appearance period of the target object to be added after being adjusted along the adjustment direction overlaps with an initial appearance period of one or more objects to be added in the storyboard in the adjustment direction, determining that the predetermined period adjustment condition is satisfied in response to a determination that an initial appearance period of the target added object after being adjusted along the adjustment direction does not overlap with an initial appearance period of one or more added objects of the storyboard in the adjustment direction; or in response to a determination that an initial appearance period of the target object to be added after being adjusted along the adjustment direction overlaps with an initial appearance period of one or more objects to be added of the storyboard in the adjustment direction, determining whether or not the initial appearance period of the target object to be added after being adjusted along the direction opposite to the adjustment direction overlaps with an initial appearance period of an object to be added of the storyboard in the direction opposite to the adjustment direction; and in response to a determination that an initial appearance period of the target object to be added after being adjusted along the direction opposite to the adjustment direction does not overlap with an initial appearance period of an object to be added of the storyboard in the direction opposite to the adjustment direction, determining that the preset period adjustment condition is satisfied. The method of claim 2.

4. The step of determining whether an initial appearance period of the target added object after being adjusted along the adjustment direction overlaps with an initial appearance period of one or more added objects of the storyboard in the adjustment direction, determining an appearance interval between the target object and the target object in the adjustment direction that is closest to the initial appearance period of the target object; 4. The method of claim 3, further comprising: determining whether an initial appearance period of the target object to be added after being adjusted along the adjustment direction overlaps with an initial appearance period of one or more objects to be added of a storyboard in the adjustment direction based on a comparison of the appearance interval and a duration indicated from an initial appearance period of the target object to be added.

5. The step of determining whether an initial appearance period of the target added object after being adjusted along the adjustment direction overlaps with an initial appearance period of one or more added objects of the storyboard in the adjustment direction, 4. The method of claim 3, further comprising: determining a total duration of an initial appearance period of an object to be added corresponding to a target storyboard, the target storyboard coinciding with the adjustment direction and overlapping with an initial appearance period of the target object to be added; and determining, based on the total duration and the duration of a period corresponding to the target storyboard, whether an initial appearance period of the target object to be added after being adjusted along the adjustment direction overlaps with an initial appearance period of one or more objects to be added of the storyboard in the adjustment direction.

6. The step of adjusting the initial appearance period of the target addition object that satisfies the period adjustment condition includes: The method according to any one of claims 2 to 5, further comprising a step of adjusting initial appearance periods of the target objects to be added that satisfy the period adjustment condition in a new storyboard so that the objects to be added in the new storyboard are uniformly distributed.

7. The target addition instruction information further includes position instruction information for indicating an appearance position of the target to be added in the target set in the video frame of the target video; and The step of generating target addition instruction information based on the determined appearance period includes: extracting video frames from the target video that match the determined occurrence period; Detecting a preset category of object from the extracted video frame and generating position information of the preset category of object; obtaining a display area of ​​the target object corresponding to the target object set; determining a solid object occlusion metric when the display area is at a different position in the matching video frame based on an overlap between the display area and a position indicated by generated position information of a preset category of object; determining a location in the video frame where the object to be added matches based on the solid occlusion metric; The method according to any one of claims 1 to 6, further comprising the step of generating target addition instruction information based on the determined appearance period and appearance position.

8. The preset category objects include face images and text images; and The step of determining a solid object occlusion metric when the display area is at a different position of the matching video frame based on an overlap degree between the display area and a position indicated by generated position information of a target of a preset category, determining a first convolution metric between the display area and a position indicated by position information of a generated face image; determining a second convolution metric between the display area and a position indicated by position information of a generated text image; and determining a solid occlusion metric when the display area is at different positions in the matching video frames based on the generated first and second overlay metrics.

9. The step of acquiring a display area of ​​the target object corresponding to the target object set includes: receiving an aspect ratio and a minimum width of an object to be added transmitted from a user; A step of obtaining the number of characters included in the target to be added; generating a character area to be added based on the number of characters and a preset font size; A method as described in claim 7 or 8, comprising a step of generating a display area of ​​the object to be added based on the character area, wherein the aspect ratio of the display area matches the aspect ratio of the object to be added transmitted from the user side, and the width of the display area is greater than or equal to the minimum width.

10. The step of determining an appearance position in the video frame of the matching object to be added based on the solid occlusion metric comprises: obtaining a positional bias metric of the object to be added, the positional bias metric including at least one of a user positional bias metric, a positional saliency metric of the object to be added, and an appearance positional rule metric of the object to be added; generating a position evaluation metric when the display area is in different positions in the matching video frames based on the solid occlusion metric and the position bias metric; A method according to any one of claims 7 to 9, comprising determining the position corresponding to the maximum value of the position evaluation metric as the position of appearance of the object in the matching video frame.

11. An acquisition unit configured to acquire a target video and an added object set to be added to the target video; a first determining unit configured to determine a period corresponding to each storyboard of the target video; a second determination unit configured to determine an appearance period of an object to be added in the set of objects to be added in the target video based on a period corresponding to each of the storyboards; A generating unit configured to generate object addition instruction information based on the determined appearance period, the object addition instruction information including time presentation information for indicating an appearance period of the object to be added in the set of objects to be added in the target video; The second determination unit includes a first determination subunit, a second determination subunit, a selection subunit, and an adjustment subunit; The first determining subunit is configured to determine a total duration of the target video; The second determining subunit is configured to determine an initial appearance period in the target video of the target object in the set of targets to be added based on a total duration of the target video; The selection subunit is configured to select an object to be added across storyboards from the set of objects to be added based on matching between the determined initial appearance period and a period corresponding to each storyboard of the target video, to generate a target set of objects to be added that are adjustment targets; The adjustment subunit is configured to set an initial appearance period of the target object not spanning a storyboard to an appearance period of the target object not spanning a storyboard in the target video, adjust an initial appearance period of the target object in the target target object set so as not to span the storyboard of the target video, and set the initial appearance period after the adjustment to an appearance period of the target object in the target video. An apparatus for determining the method of addition of an object.

12. one or more processors; and a storage device storing a program for causing the one or more processors to execute the method according to any one of claims 1 to 10.

13. A computer-readable storage medium having stored thereon a computer program for causing a computer processor to carry out the method according to any one of claims 1 to 10.

14. A computer program causing a computer to carry out the method according to any one of claims 1 to 10.

Images