Video processing method, device, electronic device and storage medium

Abstract

An embodiment of the present disclosure includes steps of sequentially collecting video frames to be processed in response to a special effect trigger operation; when a contact point is detected on the display interface, determining a target display position of the contact point in the video frame to be processed, and adding a target operation object to the target display position; when it is detected that a special effect playback condition is satisfied, sequentially determining a presentation form of the target operation object; and controlling the target operation object to be presented in the video frame to be processed according to the corresponding presentation form.

Description

[Technical Field]

[0001] [CROSS-REFERENCE TO RELATED APPLICATIONS] This application claims the benefit of priority from Chinese Patent Application No. 202210862523.1, filed with the China Patent Office on July 20, 2022, the entire contents of which are incorporated herein by reference.

[0002] [Technical field] TECHNICAL FIELD Embodiments of the present disclosure relate to the field of image processing technology, for example, to video processing methods, devices, electronic devices, and storage media. [Background technology]

[0003] With the development of network technology, more and more applications have entered into users' lives, such as a series of software that allows users to shoot short videos, and are loved by users.

[0004] To enhance the fun of video shooting, software developers can develop various special effect tools, but the special effect tools provided to users are very limited, and the richness of video content needs to be further improved. For example, in the augmented reality (AR) field, the special effect videos generated based on related special effect tools have poor results and do not have strong user interaction, resulting in a poor user experience. Summary of the Invention [Means for solving the problem]

[0005] The present disclosure provides video processing methods, devices, electronic equipment, and storage media for achieving technical effects that enhance user interactivity with special effect video content.

[0006] An embodiment of the present disclosure includes: sequentially collecting video frames to be processed in response to a special effect trigger operation; when a contact point is detected on a display interface, determining a target display position of the contact point in the video frames to be processed; and adding a target operation object to the target display position; When it is detected that the special effect reproduction conditions are satisfied, a video processing method is provided, which includes sequentially determining a presentation form of the target operation object, and controlling the target operation object so that it is presented in the processing target video frame according to the corresponding presentation form.

[0007] The present invention provides a video processing device including: a video frame collection module configured to sequentially collect video frames to be processed in response to a special effect trigger operation; an object addition module configured to determine a target display position of the contact in the video frame to be processed when a contact is detected on a display interface and add a target operation target to the target display position; a presentation form determination module configured to sequentially determine a presentation form of the target operation target when it is detected that a special effect playback condition is satisfied; and a special effect presentation module for controlling the target operation target to be displayed in the video frame to be processed according to the corresponding presentation form.

[0008] An electronic device is provided, comprising at least one processor and a storage device configured to store at least one program, wherein when the at least one program is executed by the at least one processor, the at least one processor implements a video processing method according to any of the embodiments of the present disclosure.

[0009] A storage medium including computer-executable instructions, which when executed by a computer processor, are used to perform the video processing method of any of the embodiments of the present disclosure. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a flowchart of a video processing method provided by an embodiment of the present disclosure.

[0011] [Figure 2] 1 is a flowchart of a video processing method provided by an embodiment of the present disclosure.

[0012] [Figure 3] 1 is a schematic diagram of the structure of a video processing device provided by an embodiment of the present disclosure;

[0013] [Figure 4] 1 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0014]

[0023] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. While the drawings show several embodiments of the present disclosure, it should be understood that the present disclosure may be embodied in various forms and should not be construed as being limited to the embodiments set forth herein.

[0015] It should be understood that the steps described in the method embodiments of the present disclosure may be performed in a different order and / or in parallel. Additionally, method embodiments may include additional steps and / or omit performing steps as illustrated.

[0016] As used herein, the term "comprises" and variations thereof are open inclusions, including but not limited to. The term "based on" means "based at least in part on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one other embodiment," and the term "some embodiments" means "at least some embodiments." Relevant definitions of other terms are provided in the description below.

[0017] It should be noted that the concepts of "first," "second," etc. referred to in this disclosure are used only to distinguish between different devices, modules, or units, and are not used to limit the order or interdependence of functions performed by these devices, modules, or units.

[0018] Those skilled in the art will understand that the modifications "one" and "multiple" referred to in this disclosure should be understood as "one or more" unless the context expressly indicates otherwise.

[0019] The names of messages or information interacting between multiple devices in the embodiments of the present disclosure are used for illustrative purposes only.

[0020] It will be understood that before using the technical solutions disclosed in the multiple embodiments of the present disclosure, the type, scope of use, use scenarios, etc. of personal information related to the present disclosure must be notified to the user in an appropriate manner in accordance with relevant laws and regulations, and the user's permission must be obtained.

[0021] For example, in response to receiving a user's proactive request, prompt information is sent to the user to explicitly prompt the user that the operation requested to be performed requires obtaining and using the user's personal information. Therefore, based on the presented information, the user can autonomously choose whether to provide personal information to software or hardware, such as an electronic device, an application, a server, or a storage medium, that performs the operation of the technical solution of the present disclosure.

[0022] In an alternative, non-limiting embodiment, the method for transmitting the offered information to the user in response to receiving the user's proactive request may be, for example, a pop-up method in which the offered information may be presented in text form, and the pop-up window may also include a selection control for the user to select "Agree" or "Disagree" to provide personal information to the electronic device.

[0023] It is understood that the above notification and user authentication acquisition process is merely exemplary and is not limited to the configuration of the embodiment of the present disclosure, and methods that satisfy other relevant laws and regulations may also be applied to the embodiment of the present disclosure.

[0024] It is understood that the data related to this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of applicable laws, regulations and related provisions.

[0025] Before introducing the technical solution, application scenarios can be described by way of example. For example, the technical solution provided by the embodiments of the present disclosure can increase the interaction between the user and the display interface in a scene where an AR special effect video is generated, and improve the personalization of the special effect video and the interaction effect with the user. In addition, in a scene where any special effect video in the related art is generated, a special effect video in which dominoes or any blocks in the scene are continuously knocked down can be simulated and displayed.

[0026] 1 is a flowchart of a video processing method provided by an embodiment of the present disclosure. This embodiment is suitable for a user to use a special effect tool to generate a corresponding special effect video. This method can be performed by a video processing device. This device can be implemented in the form of software and / or hardware, and the hardware can be an electronic device such as a mobile terminal, a personal computer (PC) terminal, or a server. A scene for presenting a special effect video is usually realized through the cooperation of a client and a server, and the method provided by this embodiment can be executed by the server, the client, or the client and the server in cooperation.

[0027] As shown in FIG. 1, the method includes the following steps:

[0028] S110, in response to a special effect trigger operation, sequentially collect video frames to be processed.

[0029] An apparatus for executing the special effect video processing method provided by the embodiments of the present disclosure may be integrated with application software supporting special effect video processing functions, and the software may be installed in an electronic device, which may optionally be a mobile terminal or a PC terminal. The application software may be a type of software for image / video processing, as long as it is capable of image / video processing, and description thereof is omitted here. It may also be software for adding and displaying special effects, or an application developed for integration into an appropriate page, and a user may process the special effect video using the integrated page on the PC.

[0030] In this embodiment, in application software or an application supporting a special effect video processing function, a control can be developed in advance to trigger a special effect, which starts collecting video frames to be processed in response to a special effect trigger operation when a user trigger is detected, in response to the special effect trigger operation, that is, a special effect tool corresponding to the embodiment of the present disclosure can be created, and when the special effect tool detects that it has been triggered, video frames to be processed can be collected.

[0031] After triggering the corresponding special effect tool, the user can call the imaging device to collect the video frames to be processed based on the imaging device, i.e., capture the video frames to be processed based on the imaging device.

[0032] S120: when a contact point is detected on the display interface, a target display position of the contact point in the video frame to be processed is determined, and a target operation object is added to the target display position.

[0033] The contact point is the touch point where the user places the specified target operation object on the display interface after triggering a special effect in the application. This touch point may also be a key associated with the user's own characteristics. For example, the point may be the tip of the user's nose. The current position information of the contact point, i.e., data determined by the application software or application for the position of the contact point within the display interface, such as the two-dimensional coordinates of the contact point within the display interface, is also included.

[0034] Preferably, for the triggered special effect, the user can use facial features such as a nose to determine the display interface as a contact point for determining to add a target operation object, and in a subsequent process, the user can move the display interface as the contact point based on the facial features, thereby determining a target display position of the contact point on the display interface. Alternatively, the user can preferably specify a target display position for placing the target operation object on the display interface by a touch operation of a finger on the display interface, and then place the target operation object corresponding to the target display position.

[0035] The target operation object can be understood as an object placed at the touch point, and may be, for example, a block-shaped object such as a domino.

[0036] If it is detected that the display interface includes a contact point, a target display position of the contact point in the video frame to be processed is determined, and a target operation object is added to the target display position.

[0037] In this embodiment, determining a target display position of a contact point in a video frame to be processed includes determining the contact point when a trigger operation in a display interface is detected, and determining a target display position based on the contact point.

[0038] The trigger operation may be a trigger operation by a finger on the display interface, or the point may be a stop point on the display interface. The target display position may be a position where a target operation object is placed in an AR scene.

[0039] For example, the user's nose may be preset as a target point, and based on this, when the user collects a facial image through the front camera of the mobile device, a feature in the facial image may be identified based on a pre-trained face identification algorithm. When a nose feature that can be a target key is identified in the facial image, the length of time that the feature is lingering on the display interface may be measured, and when the length of time that the feature is lingering on the display interface reaches a preset lingering time threshold, for example, 200 ms may be selected, that is, the coordinates of the feature in the display interface may be set as the target display position. Preferably, a trigger operation of a touch point on the display interface may be detected, and pressure values corresponding to multiple positions on the display interface may be detected, and a position corresponding to a pressure value higher than a preset pressure value threshold may be set as the touch point, and that position may be set as the target display position.

[0040] It should be understood that the user can click the display screen according to actual needs to determine the target display position of the touch point in the video frame to be processed on the click display screen, and place the preset target operation object at the target display position. If the number of touch points includes more than one, it means that the number of target operation objects also includes more than one.

[0041] S130: if it is detected that the special effect reproduction condition is satisfied, the presentation form of the target operation object is determined sequentially.

[0042] The special effect playback condition can be understood as a condition that requires controlling at least one placed target operation object to play an animation. The presentation form can include an upright state or a tiling state of the target operation object. When placing the target operation object based on the target display position, the state that the target operation object corresponds to is defined as the upright state, and the tiling state is defined as a state different from the upright state.

[0043] It should be understood that when it is detected that the special effect playback conditions are met, the presentation form of the target operation object can be sequentially determined, and after the presentation form is determined, the target operation object can be sequentially controlled to be presented according to its corresponding presentation form.

[0044] The presentation forms of the target operation objects can be determined sequentially based on their display positions in the display interface, for example, by combining with coordinate information of the target operation objects in the display interface to determine the order information of the presentation forms of the target operation objects.Furthermore, the display order information of the target operation objects can be determined based on the placement timestamps of the target operation objects in the display interface, and the presentation forms of the corresponding target operation objects can also be determined.

[0045] In an embodiment of the present disclosure, the special effect playing condition is that the time for adding the target operation object reaches a second set time, and triggers the control for which the addition of the target operation object has been completed.

[0046] A time threshold for adding the target operation object, such as 30 seconds or 1 minute, can be preset, and this time threshold can be understood to be the second preset time. During collection of the video frames to be processed, a countdown module can be activated to add the target operation object to the video frames to be processed within the second preset time. When the countdown reaches 0, it indicates that the target operation object has been added, and the target operation object can be controlled to be displayed with special effects according to a specified presentation format. Alternatively, the user can add the target operation object to the video frames to be processed according to their actual needs. After the addition is complete, the user can trigger a completion control on the display interface to determine that the target operation object has been added when the trigger completion control is detected. The target operation object can then be displayed according to the specified presentation format, resulting in a video using special effects.

[0047] The above method allows items to be placed within a limited time, or allows the user to have sufficient time to place the items, thereby realizing the effect of controllability over the time required to place items in an actual scene.

[0048] Furthermore, when adding a target operation object to a processing target video frame, the target operation object is used as foreground information and the sequentially collected processing target video frames are used as background information to obtain a final special effect video.

[0049] S140, the target operation object is controlled to be presented in the video frame to be processed according to the corresponding presentation format.

[0050] It is understood that when determining the presentation form corresponding to the corresponding target operation object, the object can be presented according to the determined presentation form.

[0051] For example, the target operation object is a building block or domino arranged with a certain regularity or randomly. When it is detected that the special effect playback conditions are met, the presentation form of the building block can be determined starting from the first building block. If the presentation form is a tiling form, the first building block is controlled to change from an upright state to a tiling state, and then the presentation form of the second building block is determined, and by sequential analogy, a special effect video is obtained, preferably achieving the effect of simulating dominoes in the display scene. Of course, if the presentation form of the first building block is an upright state, it is determined that the first building block does not have a corresponding effect on the second building block, and all subsequent building blocks are treated as upright. Of course, if the initial building block is not the first but the nth, the presentation form of the nth building block can be determined, and based on the presentation form of the nth building block, the presentation forms of the n+1, n+2, ... building blocks can be determined up to the final building block.

[0052] The technical solution provided by the embodiments of the present disclosure sequentially collects video frames to be processed in response to a special effect trigger operation; when a user trigger display interface is detected, the contact can determine a target display position in the video frame to be processed, and add a target operation object to the target display position; when it is detected that the special effect playback conditions are met, the presentation form of the target operation object can be sequentially determined, and the target operation object is controlled to be presented in the video frame to be processed according to the corresponding presentation form; and the placement position of the target operation object can be determined according to the user's trigger operation, that is, to achieve the effect of interaction between the user and the display interface; and further, upon meeting the special effect playback conditions, the controllable target operation object can be presented according to the corresponding presentation form in sequence, which has the effect of enhancing the richness of the screen content.

[0053] 2 is a flowchart of a video processing method provided by an embodiment of the present disclosure. Based on the above embodiment, not only can it realize special effect interaction in normal video, but also can realize special effect interaction in AR scene, for example, when collecting video frames to be processed, it can locate a target virtual plane in the video frames to be processed, and add a target operation object to the same target virtual plane according to the touch point, so as to realize the fusion effect between the target operation object and the video frames to be processed, for which embodiment please refer to the description of the embodiment of the present disclosure, where the same or corresponding technical terms as those in the above embodiment will not be further described herein.

[0054] As shown in FIG. 2, the method includes the following steps:

[0055] S210, in response to a special effect trigger operation, sequentially collect video frames to be processed.

[0056] It should be appreciated that video frames for processing can be sequentially collected upon detecting a control corresponding to a special effects tool trigger, or alternatively, video frames for processing can be sequentially collected after detecting a control for a start trigger.

[0057] In this embodiment, the method further includes: in the process of sequentially collecting the video frames to be processed, displaying at least one selection target trajectory; taking the selection target trajectory that is last triggered within a first set time as the target trajectory; displaying a guide diagram corresponding to the target trajectory on the display interface; and determining the target display position of the target operation target based on the guide diagram by the contact point; or, if at least one selection target trajectory is not triggered within the first set time, removing the at least one selection target trajectory from the video frames to be processed.

[0058] There may be one or more selection target paths, and each selection target path may display a display effect of an associated pattern. When a user triggers a special effect, the application may display a corresponding character pattern selection target path 1 and a corresponding image pattern selection target path 2 on the associated display interface. In this embodiment, the user is provided with a selection target path associated with the special effect, and a template selection countdown is displayed on the display screen, with the countdown time being a preset selection time. The user can select from multiple selection target path templates by triggering the operation. When the selection time reaches a first preset time, i.e., when the template selection countdown reaches zero, the user can understand that the selection target path selected by the user at this time is the target track. Of course, if the preset time has not yet been reached, the user can issue a confirmation command using the confirmation control, thereby directly terminating the countdown of the template selection and setting the currently selected selection target path as the target path. Those skilled in the art should understand that, when there are multiple selection target path templates, the user can select multiple templates sequentially at will, but after the selected time reaches the preset time, the application will simply set the selection target path last selected by the user as the target path. On the other hand, the method of determining the target trajectory can be selected according to the actual situation, and the embodiment of the present disclosure is not limited thereto. Setting at least one selection trajectory can guide the user to arrange content in a more visually appealing way, and further enhance the interactivity between the user and the display interface.

[0059] Of course, if the selection target trajectory is not selected within the first set time, it indicates that the user needs to present the item according to the trajectory, and in this case, the selection target trajectory can be removed from the video frame to be processed so that the user can exert himself according to his needs and determine the placement position of the target operation object.

[0060] When the user selects a target trajectory, a guide diagram corresponding to the target trajectory can be displayed.

[0061] The guide diagram is a guide diagram for guiding the user and assisting the user in locating the corresponding target operation object, and may be displayed on the display interface in the form of a dotted line, and a corresponding arrow is displayed on the guide diagram to guide the user to locate the corresponding target operation object based on the arrow.

[0062] In this embodiment, a pocket policy is also provided in advance for cases where the user has not made a selection. If at least one selection target trajectory is not selected within a set time, the selection target trajectory is deleted from the video frame to be processed. If no selection target trajectory is selected within a set time, the user can clear the selection target trajectory in the display interface. In this case, the user can understand that he or she can arrange the target operation target in any shape within the video frame to be processed at his or her own will.

[0063] S220, based on the obtained spatial position information of the terminal device, determine a target virtual plane located in the processing target video, so as to add a target operation object to the target virtual plane based on the contact point.

[0064] The terminal device refers to a device corresponding to the collection of the video frames to be processed. The spatial position information may be position information in a world coordinate system. A gyroscope and / or an inertial measurement unit installed in the terminal device can determine angle information of a target terminal device and multiple planes formed by coordinate axes in the world coordinate system. Based on the spatial position information and angle information of the terminal device, the terminal device determines a corresponding virtual plane, which is set as the target virtual plane.

[0065] When a contact point is detected on the display interface, the contact point can be positioned at a display position within the video frame to be processed, and the target operation object can be positioned on the target virtual plane. By setting in this way, all the target operation objects can be adjusted to be positioned on the same horizontal plane, priority information for the target operation objects can be determined, and the corresponding presentation form of the target operation object can be determined based on the priority information.

[0066] S230, if a contact point is detected on the display interface, determine that the contact point is at a target display position on the target virtual plane in the video frame to be processed, and add a target operation object at the target display position.

[0067] A user can trigger any location on the interface to be displayed, and this trigger point can be used as a contact point. This contact point can be determined as a target display position on a target virtual plane in a video frame to be processed, and a target operation target can be added to the display position. The number of target operation targets included depends on the number of touch points, that is, the number of touch points and the number of target operation points are the same.

[0068] In practical application, there may be a problem that the placement position of the target operation object does not match the actual need, so that an undo function can be set to solve such a problem, and the target operation object can be removed from the video frame to be processed based on the undo function.

[0069] When it is detected that a trigger operation on the target operation object satisfies a cancellation condition, the target operation object is preferably removed from the video frame to be processed, where the cancellation condition includes a trigger operation that is at least one of an operation of clicking an undo control, an operation of double-clicking the target operation object, and an operation of long-pressing the target operation object.

[0070] It can be understood that a trigger operation on a target operation object can be detected in real time, and if the trigger operation on the target object satisfies an object cancellation condition, the corresponding target operation object can be removed from the video frame being processed.

[0071] In an embodiment of the present disclosure, the cancel condition is that the display interface includes a control corresponding to an undo function, and if the undo control is triggered after detecting that the addition of the target operation object is complete, the just-added target operation object is deleted from the video frame being processed. When the control is canceled by double-clicking, the corresponding target operation objects can be canceled in order based on the timestamps at which the target operation objects were added. Furthermore, if the undo control is triggered again after the trigger of the target operation object is detected, the triggered target operation object can be removed from the video frame being processed. Furthermore, if the target operation object is double-clicked or if it is detected that the trigger time for the target operation object has reached the corresponding preset press time, it indicates that the target operation object triggered at this time is to be removed from the video frame being processed. This setting allows an appropriate target operation object to be set according to actual requirements to improve interactivity with the display interface.

[0072] S240: if it is detected that the special effect reproduction condition is satisfied, the presentation form of the target operation object is determined sequentially.

[0073] In an embodiment of the present disclosure, before sequentially determining the presentation form of the target operation object, a motion attribute corresponding to a preset target operation object is determined, and the presentation form of the target operation object is sequentially determined based on the motion attribute, or a slide operation on the display screen is determined, the motion direction of the target operation object is determined based on the slide operation, and the presentation form of the target operation object is sequentially determined based on the motion direction.

[0074] The movement attribute includes at least two embodiments in which the presentation form for determining the target operation object is determined based on a pre-set movement attribute, and the first embodiment can be understood as being for determining the presentation form of multiple target operation objects based on a trigger operation on the user's display interface.

[0075] Next, we will introduce how the presentation format is determined for each of the two implementation methods.

[0076] The motion attributes may be the motion direction corresponding to the set of target manipulation objects set in the development stage and the initial motion speed of the first target manipulation object at the start position. The presentation form may be an upright form or a tiling form. When the target manipulation objects corresponding to all target display positions are arranged, the initial form of all target manipulation objects is an upright form.

[0077] If it is detected that the special effect playback conditions are met, a starting target control object can be determined from among all target control objects. Based on a preset initial movement speed and movement direction, it is determined that the presentation form of the starting target control object will change from an upright state to a tiling state. Next, the presentation form of the second target control object can be determined. In this case, the presentation form of the second target control object can be determined based on the existing conservation of momentum or conservation of energy and the interval information between the first and second target control objects. Therefore, the presentation form of the third target control object is related to the presentation form of the second target control object. That is, the presentation form of the next target control object can be determined based on the presentation form of the previous target control object.

[0078] In this way, the effect of dominoes falling in sequence can be simulated in the presentation environment.

[0079] To further enhance the interactivity between the user and the display interface, the presentation form of the corresponding target operation object can be determined based on the user's slide operation on the display interface.

[0080] The slide operation may be a slide of a finger on the display screen, for example, a slide to the left or a slide to the right.

[0081] When the special effect playback conditions are met and a slide operation on the display screen is detected, a slide direction corresponding to the slide operation is determined, and an initial speed corresponding to the slide operation is determined based on the end point corresponding to the slide operation. A first target operation object is determined from multiple target operation objects based on the slide direction. Presentation forms corresponding to the multiple target operation objects are sequentially determined based on the first target operation object, the slide direction, and the initial speed. The determination method can refer to the above description and will not be described here.

[0082] For example, if the slide operation corresponds to a slide operation biased to the left, the slide direction is determined to be leftward, and the target operation object located at the rightmost position in the video frame to be processed can be set as the first target operation object. In this case, the moving speed of the first target operation object matches the speed at the end of the slide operation, and the slide direction is the direction of movement to the left, and based on this, the presentation modes of all target operation objects can be sequentially determined.

[0083] In this embodiment, the target of the starting operation is determined, and an object is triggered in the actual environment. Then, the object is used as the starting control point to sequentially control the falling state of subsequent items, which has the effect of enhancing the realism of the special effect video content.

[0084] In actual applications, the user may trigger any one of all the target operation objects, and in order to simulate the effect in this case, the target operation object corresponding to the slide operation can be identified and used as the starting operation object for sequentially identifying multiple target operation objects.

[0085] The starting manipulation object can be understood as the first object whose presentation form needs to be determined.

[0086] It can be understood that the target operation object corresponding to the slide operation is set as the starting operation object, and the presentation form corresponding to the corresponding target operation object can be sequentially determined based on the initial speed and slide direction corresponding to the slide operation.

[0087] What is required is to determine the starting operation target, and then determine the presentation form corresponding to the corresponding target operation based on the physics engine, and further control the target operation target to be presented according to the corresponding presentation form. In this embodiment, the presentation form for determining the corresponding target operation target based on the physics engine may be such that, if the target operation target is the starting operation target, the motion information and presentation form of the target operation target are determined based on a sliding operation on the starting operation target, or the motion information and presentation form of the next target operation target adjacent to the target operation target are determined based on the motion information and motion direction of the target operation target. Here, the motion information includes the motion speed, and the presentation form includes an upright state or a tiling state.

[0088] It can be seen that the system determines a starting object based on a physics engine, determines an initial speed and a sliding direction (direction of movement) based on a sliding operation on the starting object, determines the presentation form of the starting object, and controls the starting object to be presented according to the presentation form. Next, the system determines a next target object based on the sliding direction. The system determines the presentation form of the next target object based on the distance information between the next target object and the starting object, the mass information of the starting object, and the motion information corresponding to the tiling state of the presentation form of the starting object. The system controls the next target object to be presented according to the currently determined presentation form, and determines the presentation forms of multiple target objects by analogy. Using the above method, the system determines the presentation form of the corresponding target object, simulating the effect of multiple dominoes falling continuously in a real environment, thereby enhancing the interactivity between the display interface and the user and enhancing the realism of the display screen.

[0089] In addition, if the distance information between the initial operation target and the next target operation target is greater than the height information of the target operation target itself, regardless of how large the corresponding movement speed is when the initial operation target is converted into a tiling state, the movement speed of the presentation form corresponding to the next target operation target is zero, the presentation form is an upright state, and the multiple target operation targets after the next target operation target match the state of their initial arrangement.

[0090] Furthermore, when distance information between a plurality of target operation objects is not taken into consideration, the presentation mode of the next target operation object can be determined by combining only the motion speed of the previous target operation object.

[0091] S250, the target manipulation object is controlled to be presented in the video frame to be processed according to the corresponding presentation format.

[0092] It can be understood that in the process of presenting the target operation object based on each presentation mode, the video frame to be processed is displayed as background information.

[0093] In an embodiment of the present disclosure, when collecting video frames to be processed, based on spatial position information of the terminal device, when it is detected that a contact is included in the display interface, a target display position of the contact on a target virtual plane is determined, the target operation object is placed on the target virtual plane corresponding to the target display position, and a corresponding effect video can be created in the AR scene, and the target virtual plane can be determined to achieve the effect of improving the interactivity between the user and the display interface.

[0094] 3 is a schematic diagram of the structure of a video processing device provided by an embodiment of the present disclosure. As shown in FIG. 3, the device includes: a video frame collection module 310, an object addition module 320, a presentation form determination module 330, and a special effect presentation module 340.

[0095] The video frame collection module 310 is configured to sequentially collect video frames to be processed in response to a special effect trigger operation; the object addition module 320 is configured to determine a target display position of the contact in the video frame to be processed when a contact of the display interface is detected, and add a target operation object to the target display position; the presentation form determination module 330 is configured to sequentially determine a presentation form of the target operation object when it is detected that a special effect playback condition is satisfied; and the special effect presentation module 340 is configured to control the target operation object and display it in the video frame to be processed according to the corresponding presentation form.

[0096] The technical solution provided by the embodiments of the present disclosure is to sequentially collect video frames to be processed in response to a special effect trigger operation; when a user trigger display interface is detected, the contact can determine a target display position in the video frame to be processed, and add a target operation object to the target display position; when it is detected that the special effect playback conditions are met, the presentation form of the successive target operation objects can be determined, and the target operation objects are controlled to be presented in the video frame to be processed according to the corresponding presentation form; and the placement position of the target operation objects can be determined according to the user's trigger operation, that is, to achieve the effect of interaction between the user and the display interface; and when the special effect playback conditions are met, the operation objects that can control multiple target operation conditions are presented according to the corresponding presentation form in sequence, which has the effect of enhancing the richness of the screen content.

[0097] In addition to the above technical solutions, the device further includes a virtual plane creation module configured to determine, based on spatial position information of the terminal device, a target virtual plane located in the video to be processed, so as to add the target operation object to the target virtual plane based on the contact point.

[0098] In addition to the above-mentioned multiple technical solutions, the object addition module includes a contact determination unit configured to determine the contact point when detecting a trigger operation on the display interface, and a display position determination unit configured to determine the target display position based on the contact point.

[0099] In addition to the above-mentioned multiple technical solutions, the device further includes: a trajectory display module configured to display at least one selection object trajectory; a first trajectory selection module configured to set the selection object trajectory last triggered within a first setting time as a target trajectory, and display a guide diagram corresponding to the target trajectory on a display interface so that the contact point determines a target display position of the target operation object based on the guide diagram; or a trajectory removal module configured to remove at least one selection object trajectory from a video frame to be processed if at least one selection object trajectory is not triggered within a first setting time.

[0100] In addition to the above-mentioned technical solutions, the device further includes an operation object removal module that sets the target operation object to be removed from the video frame to be processed when it is detected that a trigger operation on the target operation object satisfies a cancellation condition, where the cancellation condition includes at least one of the following operations: the trigger operation clicking a cancel control, the operation double-clicking the target operation object, and the operation of long-pressing the target operation object.

[0101] In addition to the above-mentioned technical solutions, the special effect playing condition includes: the time for adding the target operation object reaches a second set time; and triggering a control when the addition of the target operation object is completed.

[0102] In addition to the above-mentioned technical solutions, the presentation form determination module includes: a first movement attribute determination unit configured to determine movement attributes corresponding to the target operation object, which is preset to sequentially determine the presentation form of the target operation object based on the movement attributes before sequentially determining the presentation form of the target operation object; or a second movement attribute determination unit configured to determine a slide operation on the display screen, which determines the movement direction of the target operation object based on the slide operation, and sequentially determines the presentation form of the target operation object based on the movement direction.

[0103] In addition to the above-mentioned plurality of technical solutions, the device further includes a start operation target determination module, which is configured to determine a target operation target corresponding to the slide operation, and sets the target operation target corresponding to the slide operation as a start operation target for sequentially determining a presentation mode of the target operation target.

[0104] In addition to the above-mentioned technical solutions, the device further includes, when the target operation object corresponding to the slide operation is a start operation object, determining, based on the slide operation on the start operation object, motion information and a presentation form of the target operation object corresponding to the slide operation; and determining, based on the motion information and motion direction of the target operation object corresponding to the slide operation, motion information and a presentation form of a next target operation object adjacent to the target operation object corresponding to the slide operation, wherein the motion information includes motion speed, and the presentation form includes an upright state or a tiling state.

[0105] In addition to the above-mentioned multiple technical solutions, the device further includes determining, based on the motion information and motion direction of the target operation object corresponding to the slide operation, the motion information and presentation form of a next target operation object adjacent to the target operation object corresponding to the slide operation, and determining, based on the motion information corresponding to the presentation form of the target operation object corresponding to the slide operation, pitch information corresponding to the next target operation object, and mass information of the next target operation object, the motion information and corresponding presentation form of the next target operation object.

[0106] In addition to the above-mentioned technical solutions, the special effect presentation module displays the target video frame as background information during the process of presenting the target operation object according to a corresponding presentation format.

[0107] The special effect processing device provided by the embodiments of the present disclosure can execute the special effect processing method provided by any embodiment of the present disclosure, and includes functional modules corresponding to the execution method.

[0108] The multiple units and modules included in the above device are simply divided according to functional logic, but are not limited to the above divisions as long as they can realize the corresponding functions, and the names of the multiple functional units are also simply for making it easier to distinguish from each other.

[0109] FIG. 4 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Referring now to FIG. 4, a structural schematic diagram of an electronic device (e.g., the terminal device or server of FIG. 4) 400 suitable for use in implementing an embodiment of the present disclosure is shown. Terminal devices in the embodiment of the present disclosure include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, personal digital assistants (PDAs), tablets (Portable Android Devices, PADs), portable multimedia players (Portable Media Players, PMPs), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital televisions (TVs) and desktop computers. The electronic device 400 shown in FIG. 4 is merely an example.

[0110] 4, electronic device 400 may include a processing unit (e.g., a central processor, a graphics processor, etc.) 401 that can perform a number of appropriate operations and processes in accordance with a program stored in read-only memory (ROM) 402 or loaded from storage device 408 into random access memory (RAM) 403. RAM 403 also stores a number of programs and data necessary for the operation of electronic device 400. Processing unit 401, ROM 402, and RAM 403 are connected to one another via bus 404. An input / output (I / O) interface 405 is also connected to bus 404.

[0111] Generally, the following devices may be connected to the I / O interface 405: input devices 406, including a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 407, including a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 408, including magnetic tape, hard disk, etc.; and communication devices 409. The communication devices 409 may enable the electronic device 400 to communicate wirelessly or via wires to exchange data with other devices. While FIG. 4 illustrates the electronic device 400 having multiple devices, it should be understood that it is not necessary to implement or include all of the illustrated devices. Alternatively, more or fewer devices may be implemented or included.

[0112] According to an embodiment of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product including a computer program carried on a non-transitory computer-readable medium containing program code for performing the methods shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via the communication device 409, installed from the storage device 408, or installed from the ROM 402. When the computer program is executed by the processing device 401, the functions described above, which are specific to the methods of the embodiments of the present disclosure, are performed.

[0113] The names of messages or information interacting between multiple devices in the embodiments of the present disclosure are used for illustrative purposes only.

[0114] The electronic device provided by the embodiments of the present disclosure belongs to the same inventive concept as the video processing method provided by the above embodiments, and technical details not described in detail in this embodiment can be referred to the above embodiments.

[0115] An embodiment of the present disclosure provides a computer storage medium storing a computer program that, when executed by a processor, implements the video processing method provided by the above embodiment.

[0116] It should be noted that the computer-readable medium described above in this disclosure may be a computer-readable signal medium, a computer-readable storage medium, or any combination of the above. The computer-readable storage medium may be, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any combination thereof. Examples of computer-readable storage media include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, RAM, ROM, an Erasable Programmable Read-Only Memory (EPROM) or flash memory, optical fiber, a portable Compact Disc Read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program for executing or for use in connection with a system, device, or apparatus. In contrast, in this disclosure, a computer-readable signal medium may include a propagating data signal, either in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium may be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transmit a program used by or in connection with an instruction execution system, apparatus, or device. The program code contained in the computer-readable medium may be transmitted over any suitable medium, including, but not limited to, electrical wire, optical cable, radio frequency (RF), etc., or any suitable combination of the foregoing.

[0117] In some embodiments, the clients and servers communicate using any now known or future developed network protocol, such as HyperText Transfer Protocol (HTTP), and may interconnect with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), the Internet (e.g., the Internet), and an end-to-end network (e.g., an ad hoc end-to-end network), as well as any now known or future developed network.

[0118] The computer-readable medium can be included in the electronic device, or can exist separately from the electronic device.

[0119] The computer-readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device sequentially collects video frames to be processed in response to a special effect trigger operation; when a contact is detected on the display interface, determines a target display position of the contact in the video frame to be processed and adds a target operation object to the target display position; when it is detected that a special effect playback condition is met, sequentially determines a presentation form of the target operation object; and controls the target operation object to be displayed in the video frame to be processed according to the corresponding presentation form.

[0120] Computer program code for carrying out the operations of the present disclosure can be written in one or more programming languages, or combinations thereof, including, but not limited to, object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as the "C" language. The program code can run entirely on the user computer, partially on the user computer, as a separate package, partially on the user computer and partially on a remote computer, or entirely on a remote computer or server. When referring to a remote computer, the remote computer can be connected to the user computer via any type of network, including a LAN or WAN, or can be connected to an external computer (e.g., connected via the Internet using an Internet Service Provider).

[0121] The flowcharts and block diagrams in the figures illustrate possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams may represent a module, program segment, or portion of code, including one or more executable instructions for implementing a given logical function. It should also be noted that in alternative implementations, the functions displayed in the blocks may occur in a different order than the order displayed in the figures. For example, two blocks shown in succession may actually be executed essentially in parallel or may be executed in the reverse order depending on the functionality involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented in a dedicated hardware-based system that performs a given function or operation, or may be implemented in a combination of dedicated hardware and computer instructions.

[0122] The units described in the embodiments of the present disclosure may be realized by software or hardware. The names of the units do not necessarily limit the units themselves, for example, the first acquisition unit may be described as "a unit for acquiring at least two Internet Protocol addresses."

[0123] The functions described herein may be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that may be used, but are not limited to, are Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), and proprietary standard products. This includes ASSPs (Assembled Specific Standard Parts), System on Chip (SOC), and Complex Programmable Logic Devices (CPLD).

[0124] In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain or store a program for use by, or in connection with, an instruction execution system, device, or apparatus. A machine-readable medium may also be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include one or more wire-based electrical connections, a portable computer disk, a hard disk, RAM, ROM, EPROM, or flash memory, optical fiber, CD-ROM, optical storage device, magnetic storage device, or any suitable combination of the foregoing.

[0125] The above description merely describes the embodiments of the present disclosure and the technical principles used. Those skilled in the art should understand that the scope of the present disclosure is not limited to the technical proposals based on the specific combinations of the above-mentioned technical features, but also encompasses other technical proposals formed by any combination of the above-mentioned technical features or equivalent features without departing from the concept of the above-mentioned disclosure. For example, the above-mentioned features and technical features having similar functions disclosed in the present disclosure (but not limited to) are formed by replacing each other.

[0126] Additionally, although operations are depicted in a particular order, this should not be construed as requiring these operations to be performed in the particular order or sequence depicted. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, the description above includes multiple implementation details. Certain features that are described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, multiple features that are described in the context of a single embodiment may also be implemented in multiple embodiments alone or in any suitable subcombination.

Claims

1. sequentially acquiring video frames to be processed in response to a special effect trigger operation; When a contact point is detected on the display interface, determining a target display position of the contact point in the processing target video frame, and adding a target operation target to the target display position; When it is detected that a special effect reproduction condition is satisfied, sequentially determining a presentation form of the target operation object; and controlling the target object to be presented in the processing object video frame according to a corresponding presentation form.

2. The video processing method of claim 1, further comprising: determining a target virtual plane located in the video to be processed based on the obtained spatial position information of the terminal device, in order to add the target operation object to the target virtual plane based on the contact point.

3. Determining a target display position of the contact point in the current video frame includes: Identifying the contact point when a trigger operation on the display interface is detected; The video processing method of claim 1 , further comprising determining the target display position based on the contact point.

4. In the process of sequentially collecting the video frames to be processed, displaying at least one selection target trajectory; 2. The video processing method of claim 1, further comprising: determining a target display position of the target operation target based on a guide diagram by displaying a guide diagram corresponding to the target selection object trajectory that was last triggered within a first set time; or, if the at least one selection object trajectory is not triggered within the first set time, removing the at least one selection object trajectory from the video frame to be processed.

5. When it is detected that the trigger operation on the target operation object satisfies a cancellation condition, removing the target operation object from the processing-object video frame; The video processing method of claim 1 , wherein the cancellation condition further includes that the trigger operation includes at least one of an operation of clicking a cancel control, an operation of double-clicking the target operation object, and an operation of long-pressing the target operation object.

6. The special effect reproduction conditions are: the time for adding the target operation object reaches a second set time; The video processing method according to claim 1 , further comprising at least one control that triggers completion of adding the target operation object.

7. Before sequentially determining the display form of the target operation object, determining a predetermined movement attribute corresponding to the target operation object, and sequentially determining a presentation form of the target operation object based on the movement attribute; or The video processing method of claim 1, further comprising: determining a slide operation in the display interface; determining a movement direction of the target operation object based on the slide operation; and sequentially determining a presentation form of the target operation object based on the movement direction.

8. The video processing method according to claim 7, further comprising: identifying a target operation object corresponding to the slide operation; and setting the target operation object corresponding to the slide operation as a start operation object for sequentially identifying a presentation form of the target operation object.

9. The number of the target operation objects is plural, and sequentially determining the presentation forms of the target operation objects includes: When one of the target operation objects is a start operation object, determining motion information and a presentation form of the one target operation object based on a slide operation on the start operation object; determining the motion information and presentation form of a next target operation object adjacent to the one target operation object based on the motion information and movement direction of the one target operation object until the display forms of the plurality of target operation objects are sequentially determined; The video processing method of claim 1 , wherein the motion information includes a motion speed, and the presentation form includes an upright state or a tiling state.

10. determining, based on the motion information and the motion direction of the target operation object corresponding to the slide operation, motion information and a presentation form of a next target operation object adjacent to the target operation object corresponding to the slide operation; 10. The video processing method of claim 9, further comprising: determining the motion information and corresponding presentation form of the next target operation object based on motion information corresponding to the presentation form of the target operation object corresponding to the slide operation, pitch information corresponding to the next target operation object, and mass information of the next target operation object.

11. Controlling the target operation object to be presented in the processing-target video frame according to a corresponding presentation form includes:

2. The video processing method according to claim 1, further comprising: displaying the processing object video frame as background information in the process of presenting the target operation object according to a corresponding presentation format.

12. 1. A video processing apparatus, comprising: a video frame capture module configured to sequentially capture video frames for processing in response to a special effect trigger operation; an object addition module configured to, when a contact point is detected on a display interface, determine a target display position of the contact point in the processing target video frame, and add a target operation object to the target display position; a presentation form determination module configured to sequentially determine a presentation form of the target operation object when it is detected that a special effect reproduction condition is satisfied; a special effect presentation module for controlling the target operation object to be displayed in the processing target video frame according to a corresponding presentation format; 2. A video processing device comprising:

13. An electronic device, at least one processor; a storage device configured to store at least one program; An electronic device, wherein said at least one program, when executed by said at least one processor, causes said at least one processor to implement the video processing method of any one of claims 1 to 11.

14. A storage medium containing computer-executable instructions, The computer-executable instructions, when executed by a computer processor, are used to perform the video processing method of any one of claims 1 to 11. storage medium.

Images