Video processing method and device, electronic equipment and storage medium

Abstract

The method comprises: in response to a special effect triggering operation, sequentially collecting to-be-processed video frames; when a touch point on a display interface is detected, determining a target display position of the touch point in the to-be-processed video frames, and adding a target control object at the target display position; when a special effect playing condition is detected, sequentially determining display forms of the target control object; and controlling the target control object to display according to the corresponding display forms in the to-be-processed video frames. The technical solution provided in the embodiments of the present disclosure can determine the placement position of the target control object based on the triggering operation of the user, that is, the interaction effect between the user and the display interface is achieved. Furthermore, on the basis of meeting the special effect playing condition, each target control object can be controlled to display according to the corresponding display form, thereby improving the richness of the picture content.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the technical field of image processing, and particularly relate to a video processing method and device, electronic equipment and storage medium. BACKGROUND

[0002] With the development of network technology, more and more application programs have entered the life of users, especially a series of software that can shoot short videos, which are deeply loved by users.

[0003] In order to improve the interestingness of video shooting, software developers can develop various special effects props, however, the special effects props provided for users are very limited, and the richness of video content needs to be further improved, especially in the field of augmented reality (AR), the special effect video effect generated based on the related special effects props is not good, and the interactivity between the special effects props and the users is not strong, thereby causing the problem of poor user experience. SUMMARY

[0004] The present disclosure provides a video processing method and device, electronic equipment and storage medium to achieve the technical effect of improving the interactivity between special effect video content and users.

[0005] In a first aspect, the embodiments of the present disclosure provide a video processing method, which comprises:

[0006] In response to a special effect triggering operation, a to-be-processed video frame is sequentially collected;

[0007] When a touch on a display interface is detected, a target display position of the touch in the to-be-processed video frame is determined, and a target control object is added at the target display position;

[0008] When a special effect playing condition is detected, a display form of the target control object is sequentially determined;

[0009] The target control object is controlled to be displayed in the to-be-processed video frame according to the corresponding display form.

[0010] In a second aspect, the embodiments of the present disclosure also provide a video processing device, which comprises:

[0011] A video frame collection module is configured to collect a to-be-processed video frame sequentially in response to a special effect triggering operation;

[0012] An object adding module is configured to determine a target display position of a touch on a display interface in the to-be-processed video frame when the touch on the display interface is detected, and add a target control object at the target display position;

[0013] The display form determination module is configured to determine, in sequence, the display form of the target control object when it is detected that the special effect playing condition is met.

[0014] The special effect display module is configured to control the target control object to display in the to-be-processed video frame according to the corresponding display form.

[0015] In a third aspect, the embodiments of the present disclosure further provide an electronic device, and the electronic device comprises:

[0016] one or more processors;

[0017] a storage device configured to store one or more programs,

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement the video processing method according to any of the embodiments of the present disclosure.

[0019] In a fourth aspect, the embodiments of the present disclosure further provide a storage medium containing computer executable instructions, which, when executed by a computer processor, are configured to perform the video processing method according to any of the embodiments of the present disclosure.

[0020] The technical solution provided by the embodiments of the present disclosure can collect to-be-processed video frames in sequence in response to a special effect triggering operation, can determine the target display position of the touch point in the to-be-processed video frame when it is detected that the user triggers the display interface, and can add a target control object at the target display position, can determine the display form of the target control object in sequence when it is detected that the special effect playing condition is met, and can control the target control object to display in the to-be-processed video frame according to the corresponding display form, thereby achieving the effect that the placement position of the target control object can be determined based on the triggering operation of the user, achieving the effect of the interaction between the user and the display interface, and further, on the basis of the special effect playing condition being met, each target control object can be controlled to display in sequence according to the corresponding display form, thereby improving the effect of the richness of the picture content. BRIEF DESCRIPTION OF DRAWINGS

[0021] The above and other features, advantages, and aspects of the embodiments of the present disclosure will become more apparent by describing in detail the following specific embodiments with reference to the attached drawings. Throughout the drawings, the same or similar reference numerals refer to the same or similar elements. It should be understood that the drawings are schematic, and the original and elements are not necessarily drawn according to the scale.

[0022] Figure 1 A video processing method flowchart provided by an embodiment of the present disclosure;

[0023] Figure 2 A video processing method flowchart provided by an embodiment of the present disclosure;

[0024] Figure 3 A structural schematic diagram of a video processing device provided by an embodiment of the present disclosure is shown in FIG. 1.

[0025] Figure 4 A structural schematic diagram of an electronic device provided by an embodiment of the present disclosure is shown in FIG. 2. DETAILED DESCRIPTION

[0026] Embodiments of the present disclosure will be described in more detail with reference to the drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein, but rather the embodiments are provided so that the present disclosure can be more thoroughly and completely understood. It should be understood that the drawings and embodiments of the present disclosure are only for illustrative purposes and are not intended to limit the scope of protection of the present disclosure.

[0027] It should be understood that each of the steps recited in the method embodiments of the present disclosure can be executed in different orders and / or in parallel. In addition, the method embodiments can include additional steps and / or omit the execution of the steps shown. The scope of the present disclosure is not limited in this respect.

[0028] The term "comprising" and variations thereof as used in the present disclosure are open-ended, that is, "including but not limited to". The term "based on" is "based, at least in part, on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Related definitions will be given in the description below.

[0029] It should be noted that the concepts of "first", "second", etc. mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not intended to limit the order or interdependence of the functions performed by these devices, modules or units.

[0030] It should be noted that the adjectives "one", "multiple" mentioned in the present disclosure are illustrative and not limiting, and those skilled in the art should understand that "one or more" should be understood unless the context clearly indicates otherwise.

[0031] The names of the messages or information exchanged between the devices in the embodiments of the present disclosure are only for illustrative purposes and are not intended to limit the scope of the messages or information.

[0032] It can be understood that, before using the technical solutions disclosed in the embodiments of the present disclosure, the type, scope of use, use scenario, etc. of the personal information involved in the present disclosure should be informed to the user and the authorization of the user should be obtained in accordance with relevant laws and regulations.

[0033] For example, in response to receiving an active request of a user, a prompt information is sent to the user to explicitly prompt the user that the operation requested to be performed will require obtaining and using personal information of the user. Thus, the user can autonomously select whether to provide the personal information to the software or hardware, such as an electronic device, an application program, a server or a storage medium, performing the operation of the technical solution of the present disclosure according to the prompt information.

[0034] As an optional but non-limiting implementation manner, in response to receiving an active request of a user, the manner of sending a prompt information to the user may, for example, be a pop-up window manner, in which the prompt information can be presented in a text manner. In addition, the pop-up window can also carry a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0035] It can be understood that the above notification and obtaining user authorization process is only illustrative and does not limit the implementation manner of the present disclosure, and other manners meeting relevant laws and regulations can also be applied to the implementation manner of the present disclosure.

[0036] It can be understood that the data involved in the technical solution (including but not limited to the data itself, the obtaining or use of the data) should comply with the requirements of relevant laws and regulations and relevant provisions.

[0037] Before introducing the technical solution, an exemplary application scenario can be described first. The technical solution provided by the embodiment of the present disclosure can be applied in any scenario requiring generation of a special effect video, for example, in the scenario of AR special effect video generation, the interactivity between the user and the display interface can be increased, and the effect of personalization of the special effect video and the interactivity with the user is improved. It can also be any existing special effect video generation scenario, which can simulate a special effect video in which bones or any block-shaped objects in the display scenario are continuously pushed down.

[0038] Figure 1 A flowchart of a video processing method provided by the embodiment of the present disclosure is shown. The embodiment can be applied to a case where a user generates a corresponding special effect video by using a special effect prop. The method can be executed by a video processing device, which can be implemented in the form of software and / or hardware. The hardware can be an electronic device, such as a mobile terminal, a PC terminal or a server, etc. The scenario of special effect video display is usually implemented by cooperation of a client and a server. The method provided by the embodiment can be executed by the server, the client or the cooperation of the client and the server.

[0039] As shown in Figure 1 the method comprises:

[0040] S110, in response to a special effect triggering operation, sequentially acquiring video frames to be processed.

[0041] The device for performing the special effect video processing method provided in the embodiments of the present disclosure can be integrated in application software supporting special effect video processing function, and the software can be installed in an electronic device. The electronic device can be a mobile terminal or a PC terminal, etc. The application software can be a kind of software for image / video processing, and specific application software will not be described here, as long as it can realize image / video processing. It can also be a specially developed application program to realize the software of adding special effects and displaying special effects, or be integrated in the corresponding page, and the user can realize the processing of the special effect video through the integrated page in the PC terminal.

[0042] In the application software or application program supporting special effect video processing function, a control for triggering special effects can be developed in advance. When it is detected that the user triggers the control, the special effect triggering operation is responded, so as to start collecting the to-be-processed video frames of the scene to which the terminal device belongs. That is, a special effect prop corresponding to the embodiments of the present disclosure can be created, and when it is detected that the special effect prop is triggered, the to-be-processed video frames can be collected.

[0043] Specifically, after the user triggers the corresponding special effect prop, the camera device can be called to collect the to-be-processed video frames based on the camera device. That is, the to-be-processed video frames are photographed based on the camera device.

[0044] S120, when the touch point on the display interface is detected, determining the target display position of the touch point in the to-be-processed video frames, and adding the target control object at the target display position.

[0045] The touch point can be a touch point determined by the user in the application software after triggering a special effect, which is used to place the target control object in the display interface. The touch point is also a key point associated with a certain feature of the user himself, for example, the key point can be the tip of the nose. The current position information of the touch point is the data determined by the application software or application program for the specific position of the touch point in the display interface, for example, the two-dimensional coordinates of the touch point in the display interface.

[0046] Optionally, for the triggered special effect, the user can determine the touch point in the display interface for adding the target control object by using a certain feature of the face (such as the nose). In the subsequent process, the user can also move the touch point in the display interface based on the facial feature as the touch point, so as to determine the target display position of the touch point in the display interface; or the user can determine the target display position of the display interface for placing the target control object by the touch operation of the finger on the display interface, and then place the corresponding target control object at the target display position.

[0047] The target operation object can be understood as an object placed at the touch point, for example, can be a block object, such as a mahjong, etc.

[0048] Specifically, when the touch point is detected in the display interface, the target display position of the touch point in the video frame to be processed is determined, and the target operation object is added at the target display position.

[0049] In the embodiment, the target display position of the touch point in the video frame to be processed is determined, including: determining the touch point when detecting the trigger operation on the display interface; determining the target display position based on the touch point.

[0050] The trigger operation can be a trigger operation of a finger on the display interface, or a key point on the display interface. The target display position can be a position of placing the target operation object in the AR scene.

[0051] For example, the user's nose is pre-set as the target key point. Based on this, when the user captures the face image through the front camera of the mobile device, the features in the face image can be recognized based on the pre-trained face recognition algorithm. When the nose feature that can be used as the target key point is recognized in the face image, the staying time of the feature in the display interface can be tracked. If the staying time reaches the pre-set staying time threshold, optionally 200ms, the coordinates of the feature in the display interface can be used as the target display position. Further, the trigger operation of the touch point on the display interface can be detected. Optionally, the pressure values corresponding to each position on the display interface are detected, and the position corresponding to the pressure value higher than the pre-set pressure value threshold is used as the touch point, and the corresponding position is used as the target display position.

[0052] It can be understood that the user can click on the display interface according to the actual demand to determine the target display position of the touch point in the video frame to be processed on the display interface, and place the pre-set target operation object at the target display position. If the number of touch points includes multiple, the number of target operation objects also includes multiple.

[0053] S130, when the special effect playing condition is detected, the display form of the target operation object is determined in sequence.

[0054] The special effect playing condition can be understood as a condition for controlling at least one placed target operation object to perform animation playing. The display form can include a standing state or a flat state of the target operation object. The state of the target operation object when the target operation object is placed based on the target display position is used as the standing state, and the flat state is a state different from the standing state.

[0055] It can be understood that when it is detected that the special effect playing condition is met, the display forms of the target control objects can be determined in sequence, and after the display forms are determined, the target control objects are controlled to be displayed in the corresponding display forms in sequence.

[0056] It should be noted that the display forms of the target control objects can be determined according to the display positions of the target control objects in the display interface, for example, the sequence information of the display forms of the target control objects can be determined in combination with the coordinate information of the target control objects in the display interface. Alternatively, the sequence information of the display forms of the target control objects can be determined according to the time stamps of the target control objects in the display interface, and then the display forms of the corresponding target control objects are determined.

[0057] In the embodiments of the present disclosure, the special effect playing condition can be that the time length of adding the target control object reaches a second preset time length, and a control for triggering the addition completion of the target control object is triggered.

[0058] It can be understood that the time length threshold for adding the target control object can be set in advance, which can be 30 seconds or 1 minute, and the time length threshold can be used as the second preset time length. During the process of collecting the to-be-processed video frame, a countdown module can be started to add the target control object in the to-be-processed video frame within the second preset time length. When the countdown is 0, it indicates that the target control object has been added, and the target control object can be controlled to be displayed in the determined display form. Alternatively, the user can add the target control object in the to-be-processed video frame according to actual needs, and after the addition is completed, a completion control on the display interface can be triggered. When it is detected that the completion control is triggered, it is determined that the target control object has been added, and the target control objects can be controlled to be displayed in the determined display form, so as to obtain the special effect video.

[0059] The above-mentioned manner has the advantages that the placement of the objects can be performed within a limited time, or the user is given sufficient time to place the objects, and the controllability of the time length of the object placement in the actual scene is realized.

[0060] It should be further noted that when the target control object is added in the to-be-processed video frame, the target control object can be used as foreground information, and the to-be-processed video frame collected in sequence can be used as background information, so as to obtain the final special effect video.

[0061] S140, the target control object is controlled to be displayed in the to-be-processed video frame according to the corresponding display form.

[0062] It can be understood that when the display form corresponding to the target control object is determined, the target control object can be displayed in the determined display form.

[0063] For example, the target control object is a building block or domino placed in a certain rule or randomly. When it is detected that the special effect playing condition is met, the display form of the first building block can be determined, and the display form is a flat form. Then, the first building block is adjusted from the upright state to the flat state. The display form of the second building block is determined, and the special effect video is obtained by analogy. The effect of simulating the display scene of the domino is achieved. Of course, if the display form of the first building block is the upright state, it is determined that the first building block will not have a corresponding influence on the second building block. All subsequent building blocks are in the upright state. Of course, if the initial building block is not the first one but the nth one, the display form of the nth building block can be determined. According to the display form of the nth building block, the display forms of the n+1, n+2, and n+3 building blocks are determined until the last building block.

[0064] The technical solution provided by the embodiment of the present disclosure responds to the special effect trigger operation, and sequentially acquires the video frames to be processed. When it is detected that the user triggers the display interface, the target display position of the touch point in the video frames to be processed can be determined, and the target control object can be added at the target display position. When it is detected that the special effect playing condition is met, the display form of the target control object can be determined in sequence, and the target control object can be controlled to be displayed in the video frames to be processed according to the corresponding display form. The display position of the target control object can be determined based on the trigger operation of the user, that is, the interaction effect between the user and the display interface is achieved. Further, based on the special effect playing condition, each target control object can be controlled to be displayed in sequence according to the corresponding display form, and the richness of the picture content is improved.

[0065] Figure 2 The video processing method flowchart provided by the embodiment of the present disclosure not only realizes special effect interaction in ordinary videos, but also realizes special effect interaction in AR scenes. For example, when the video frames to be processed are acquired, the target virtual plane located in the video frames to be processed is determined. The target control object is added on the same target virtual plane based on the touch point, so that the target control object is fused with the video frames to be processed. The specific implementation can be referred to the detailed description of the embodiment of the present disclosure. The same or corresponding technical terms as the above embodiments are not described here.

[0066] As shown in Figure 2 , the method comprises:

[0067] S210, in response to a special effect trigger operation, sequentially acquiring video frames to be processed.

[0068] It can be understood that the to-be-processed video frames can be sequentially collected when the control corresponding to the special effect prop is detected.

[0069] In the embodiment, in the process of sequentially collecting the to-be-processed video frames, the following operations are further included: displaying at least one to-be-selected track; taking the to-be-selected track that is last triggered in a first preset time length as a target track, and displaying the guide image corresponding to the target track on the display interface, so that the touch point determines the target display position of the target control object based on the guide image; or, if at least one to-be-selected track is not triggered in the first preset time length, removing the at least one to-be-selected track from the to-be-processed video frames.

[0070] In the embodiment, the to-be-selected track can have one or more, and each to-be-selected track can present the display effect of the associated pattern. When the user triggers a special effect, the application can display the to-be-selected track 1 corresponding to a certain text pattern and the to-be-selected track 2 corresponding to a certain image pattern in the related display interface. In the embodiment, while the user is provided with the to-be-selected track associated with the special effect, a template selection countdown is also displayed on the display interface, and the countdown time is the pre-set selection time length. It can be understood that the user can select a plurality of to-be-selected track templates by triggering operations, and when the selection time length reaches the first preset time length, that is, the template selection countdown is zero, the to-be-selected track selected by the user at this moment is the target track. Of course, when the pre-set time length is not reached, the user can also issue a confirmation instruction based on the determination control, so as to directly end the timing process of the template selection countdown and take the currently selected to-be-selected track as the target track. It should be understood by those skilled in the art that, on the one hand, when there are multiple to-be-selected track templates, the user can select the templates in sequence according to his own will, but after the selection time reaches the pre-set time length, the application will only take the to-be-selected track selected by the user as the target track; on the other hand, the specific way of determining the target track can be selected according to the actual situation, and the embodiment of the present disclosure does not make specific limitations here. The advantage of setting at least one to-be-selected track is that it can guide the user to place content that is more visually appealing, and further improve the interaction effect between the user and the display interface.

[0071] Of course, if the selection of the to-be-selected track is not triggered within the first preset time length, it means that the user needs to combine the track to display the object, at which time the to-be-selected track can be removed from the to-be-processed video frames, so that the user can determine the placement position of the target control object according to his own needs.

[0072] Further, if the user selects the target track, the guide image corresponding to the target track can be displayed.

[0073] The guide map is in the form of a dotted line displayed on the display interface, and there are corresponding arrows on the guide map to guide the user to place the corresponding target control object based on the arrows.

[0074] It should be noted that in this embodiment, a bottom strategy is also deployed for the case where the user does not make a selection. Specifically, if at least one to-be-selected track is not triggered within the preset time period, the to-be-selected track is removed from the to-be-processed video frame. It can be understood that if the user does not select any to-be-selected track within the preset time period, the to-be-selected track in the display interface will be emptied. In this case, the user can place the target control object in any shape in the to-be-processed video frame according to his own will.

[0075] S220, based on the obtained spatial position information of the terminal device, determining a target virtual plane located in the to-be-processed video, to add a target control object on the target virtual plane based on the touch point.

[0076] The terminal device refers to the device corresponding to the collection of the to-be-processed video frame. The spatial position information can be position information in a world coordinate system. The gyroscope and / or inertial measurement unit provided in the terminal device can determine the angle information of each plane formed by the coordinate axis and the target terminal device in the world coordinate system. According to the spatial position information of the terminal device and the angle information, the virtual plane corresponding to the terminal device is determined as the target virtual plane.

[0077] When the touch point on the display interface is detected, the target control object can be placed on the target virtual plane according to the display position of the touch point in the to-be-processed video frame. The advantage of this setting is that all target control objects are adjusted to be located on the same horizontal plane, thereby determining the sequence information of the target control objects, and then determining the display form of the target control objects according to the sequence information.

[0078] S230, when detecting the touch point on the display interface, determining the target display position of the touch point on the target virtual plane in the to-be-processed video frame, and adding the target control object at the target display position.

[0079] It can be understood that the user can trigger any position on the display interface, and the trigger point is taken as the touch point. The target display position of the touch point on the target virtual plane in the to-be-processed video frame can be determined to add the target control object at the display position. The number of touch points is consistent with the number of target control objects, that is, the number of touch points and target control points is consistent.

[0080] In actual application, there can be a problem that the placement position of the target manipulation object does not conform to actual demand. In order to solve such a problem, a withdrawal function can be set to remove the target manipulation object from the to-be-processed video frame based on the withdrawal function.

[0081] Optionally, when it is detected that the trigger operation on the target manipulation object satisfies a withdrawal condition, the target manipulation object is removed from the to-be-processed video frame; wherein the withdrawal condition includes at least one of the following: the trigger operation is an operation of clicking a withdrawal control, an operation of double-clicking the target manipulation object, and an operation of long-pressing the target manipulation object.

[0082] It can be understood that the trigger operation on the target manipulation object can be detected in real time, and if the trigger operation on the target object satisfies the object withdrawal condition, the corresponding target manipulation object can be removed from the to-be-processed video frame.

[0083] In the embodiments of the present disclosure, the withdrawal condition can be that the display interface includes a control corresponding to the withdrawal function. When it is detected that the withdrawal control is triggered after the target manipulation object is added, the target manipulation object just added is removed from the to-be-processed video frame. If the withdrawal control is clicked continuously, the corresponding target manipulation object can be removed in turn according to the time stamp of adding the target manipulation object. It can also be that when it is detected that the target manipulation object is triggered and the withdrawal control is triggered again, the target manipulation object just triggered is removed from the to-be-processed video frame. It can also be that when it is detected that the target manipulation object is double-clicked or the trigger time length of the target manipulation object reaches a corresponding preset pressing time length, it is indicated that the target manipulation object triggered at this time is removed from the to-be-processed video frame. The advantage of setting this way is that the corresponding target manipulation object can be set according to actual demand, and the effect of interaction with the display interface is improved.

[0084] S240, when it is detected that the special effect playing condition is satisfied, the display form of the target manipulation object is determined in turn.

[0085] In the embodiments of the present disclosure, before the display form of the target manipulation object is determined in turn, the motion attribute corresponding to the target manipulation object set in advance can be determined to determine the display form of the target manipulation object in turn based on the motion attribute; or, the sliding operation on the display interface is determined, and the motion direction of the target manipulation object is determined based on the sliding operation to determine the display form of the target manipulation object in turn based on the motion direction.

[0086] The motion attribute can be understood as that there are at least two implementation manners for determining the display form of the target manipulation object. The first implementation manner is determined based on the motion attribute set in advance, and the second implementation manner is to determine the display form of each target manipulation object according to the trigger operation of the user on the display interface.

[0087] Next, two ways to determine the display form are introduced.

[0088] The motion attribute can be a motion direction corresponding to each series of target control objects set in the development stage and an initial motion speed of the first target control object located at the starting position. The display form includes an upright form or a flat form. When placing the corresponding target control objects at each target display position, the initial form of each target control object is the upright form.

[0089] It can be understood that when the special effect playing condition is detected, the starting target control object in all target control objects can be determined. According to the pre-set initial motion speed and motion direction, the display form of the starting target control object is determined to change from the upright state to the flat state. Next, the display form of the second target control object is determined, at this time, the display form of the second target control object can be determined based on the existing momentum conservation or energy conservation and the distance information between the first target control object and the second target control object. Correspondingly, the display form of the third target control object is associated with the display form of the second target control object, that is, the display form of the next target control object can be determined according to the display form of the previous target control object.

[0090] The above-mentioned way has the advantage that the effect of the dominoes falling in the display environment in turn can be simulated.

[0091] In order to further improve the interactivity between the user and the display interface, the display form of the corresponding target control object can be determined based on the sliding operation of the user on the display interface.

[0092] The sliding operation can be the sliding of the finger on the display interface, for example, left-biased sliding, right-biased sliding, etc.

[0093] Specifically, when the special effect playing condition is detected and the sliding operation on the display interface is detected, the sliding direction corresponding to the sliding operation is determined, and the initial speed corresponding to the sliding operation is determined according to the end point of the sliding operation. Based on the sliding direction, the first target control object is determined from the plurality of target control objects. Based on the first target control object, the sliding direction and the initial speed, the display form corresponding to each target control object is determined in turn, and the specific determination method can be referred to the above description, which will not be repeated here.

[0094] For example, if the sliding operation corresponds to a left-biased sliding operation, the sliding direction is determined to be left, and the target operation object located at the rightmost side in the video frame to be processed can be determined as the first target operation object. At this time, the motion speed of the first target operation object is consistent with the speed at the end of the sliding operation, and the sliding direction is the direction of moving to the left. Based on this, the display mode of each target operation object can be determined in sequence.

[0095] In this embodiment, the advantage of determining the starting operation object is that the state of sequentially controlling subsequent objects to appear and fall down after triggering a certain object in a real environment can be simulated, and the real effect of the special effect video content is improved.

[0096] In actual application, there can be a case that the user triggers any one of all target operation objects. In order to simulate the effect at this time, the target operation object corresponding to the sliding operation can be determined, and the target operation object is determined as the starting operation object for sequentially determining the display mode of each target operation object.

[0097] The starting operation object can be understood as the first object whose display mode needs to be determined.

[0098] It can be understood that the target operation object corresponding to the sliding operation can be determined as the starting operation object, and the display mode of the corresponding target operation object can be determined in sequence according to the initial speed and sliding direction corresponding to the sliding operation.

[0099] It needs to be determined that after the starting operation object is determined, the display mode of the corresponding target operation object can be determined based on the physical engine, and then the target operation object is controlled to display according to the corresponding display mode. In this embodiment, the display mode of the corresponding target operation object based on the physical engine can be: if the target operation object is the starting operation object, the motion information and display mode of the target operation object are determined based on the sliding operation of the starting operation object; the motion information and display mode of the next target operation object adjacent to the target operation object are determined according to the motion information and motion direction of the target operation object; wherein the motion information includes the motion speed, and the display mode includes the upright state or the flat state.

[0100] It can be understood that the starting manipulation object is determined based on the physical engine, and the initial speed and the sliding direction (motion direction) are determined according to the sliding operation on the starting manipulation object, the display form of the starting manipulation object is determined, and the starting manipulation object is controlled to be displayed in the display form. Next, the next target manipulation object is determined according to the sliding direction, and the display form of the next target manipulation object is determined according to the distance information between the next target manipulation object and the starting manipulation object, the quality information of the starting manipulation object, and the motion information corresponding to the display form of the starting manipulation object in the flat state, and the next target manipulation object is controlled to be displayed in the display form determined at this time. The display forms of the target manipulation objects are determined in this way. The advantage of determining the display form of the corresponding target manipulation object in the above way is that the effect of the continuous falling of each mahjong in the real environment can be simulated, which not only improves the interactivity between the display interface and the user, but also improves the realism of the display picture.

[0101] It should be noted that if the distance information between the initial manipulation object and the next target manipulation object is greater than the height information of the target manipulation object itself, no matter how large the motion speed corresponding to the conversion of the initial manipulation object into the flat state is, the motion speed of the display form corresponding to the next target manipulation object is zero, and the display form is the upright state, so that each target manipulation object after the next target manipulation object is consistent with the initial placement state.

[0102] It should be noted that if the distance information between the target manipulation objects is not considered, the display form of the next target manipulation object can be determined only in combination with the motion speed of the previous target manipulation object.

[0103] S250, control the target manipulation object to be displayed in the display form in the to-be-processed video frame.

[0104] It can be understood that the target manipulation object is displayed according to the corresponding display form.

[0105] The technical scheme provided by the embodiment of the disclosure can determine the target virtual plane according to the spatial position information of the terminal device when collecting the to-be-processed video frame, so that when the display interface includes a touch point, the target display position of the touch point on the target virtual plane can be determined, and the target manipulation object can be placed on the target virtual plane corresponding to the target display position. The effect of improving the interactivity between the user and the display interface is realized in the AR scene, and the corresponding special effect video can be made.

[0106] Figure 3 A structural schematic diagram of a video processing device provided by an embodiment of the disclosure is shown in Figure 3As shown, the apparatus comprises a video frame acquisition module 310, an object adding module 320, a display form determination module 330, and a special effect display module 340.

[0107] The video frame acquisition module 310 is configured to acquire the video frames to be processed in sequence in response to a special effect triggering operation. The object adding module 320 is configured to determine a target display position of the touch point in the video frame to be processed and add a target control object at the target display position when detecting the touch on the display interface. The display form determination module 330 is configured to determine the display form of the target control object in sequence when detecting that a special effect playing condition is met. The special effect display module 340 is configured to control the target control object to display in the video frame to be processed according to the corresponding display form.

[0108] The technical solution provided by the embodiments of the present disclosure can acquire the video frames to be processed in sequence in response to a special effect triggering operation, determine the target display position of the touch point in the video frame to be processed and add a target control object at the target display position when detecting the touch on the display interface, determine the display form of the target control object in sequence when detecting that a special effect playing condition is met, and control the target control object to display in the video frame to be processed according to the corresponding display form, thereby achieving the effect that the placement position of the target control object can be determined based on the triggering operation of the user, achieving the interaction between the user and the display interface, and further improving the richness of the picture content based on the special effect playing condition.

[0109] On the basis of the above technical solution, the apparatus further comprises a virtual plane creation module configured to determine a target virtual plane in the video to be processed based on the spatial position information of the terminal device, and add the target control object on the target virtual plane based on the touch point.

[0110] On the basis of the above technical solution, the object adding module comprises:

[0111] A touch point determination unit configured to determine the touch point when detecting the triggering operation on the display interface.

[0112] A display position determination unit configured to determine the target display position based on the touch point.

[0113] On the basis of the above technical solution, the apparatus further comprises:

[0114] A track display module configured to display at least one to-be-selected track.

[0115] The first trajectory selection module is configured to select a last triggered trajectory in a first preset time period as a target trajectory, and display a guide image corresponding to the target trajectory on a display interface, so that the touch point determines a target display position of the target control object based on the guide image; or,

[0116] The trajectory removal module is configured to remove the at least one selected trajectory from the to-be-processed video frame if the at least one to-be-selected trajectory is not triggered in the first preset time period.

[0117] On the basis of the above technical solutions, the device further comprises:

[0118] The control object removal module is configured to remove the target control object from the to-be-processed video frame when it is detected that a triggering operation on the target control object satisfies a withdrawal condition, wherein the withdrawal condition comprises at least one of the following: an operation of clicking a withdrawal control, an operation of double-clicking the target control object, and an operation of long-pressing the target control object.

[0119] On the basis of the above technical solutions, the special effect playing condition comprises: a time period of adding the target control object reaches a second preset time period; and a control for triggering that the addition of the target control object is completed.

[0120] On the basis of the above technical solutions, the display form determination module is further configured to:

[0121] The first motion attribute determination unit is configured to determine a pre-set motion attribute corresponding to the target control object, and determine the display form of the target control object based on the motion attribute; or,

[0122] The second motion attribute determination unit is configured to determine a sliding operation on the display interface, determine a motion direction of the target control object based on the sliding operation, and determine the display form of the target control object based on the motion direction.

[0123] On the basis of the above technical solutions, the device further comprises a starting control object determination module configured to determine a target control object corresponding to the sliding operation, and determine the target control object as a starting control object for determining the display form of each target control object.

[0124] Based on the above technical solutions, the device further includes: if the target control object is the starting control object, then based on the sliding operation on the starting control object, determining the motion information and display form of the target control object; and determining the motion information and display form of the next target control object adjacent to the target control object according to the motion information and motion direction of the target control object; wherein the motion information includes motion speed, and the display form includes an upright state or a flat state.

[0125] Based on the above technical solutions, the device further includes: determining the motion information and display form of the next target control object adjacent to the target control object according to the motion information and motion direction of the target control object, including:

[0126] Based on the motion information corresponding to the display form of the target control object, the distance information corresponding to the next target control object, and the mass information of the next target control object, the motion information and corresponding display form of the next target control object are determined.

[0127] Based on the above technical solutions, the special effects display module displays the target manipulated object as background information during the process of displaying the target manipulated object according to its corresponding display form.

[0128] The special effects processing apparatus provided in this disclosure can execute the special effects processing methods provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects for executing the methods.

[0129] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of this disclosure.

[0130] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Reference is made below. Figure 4 It illustrates an electronic device suitable for implementing embodiments of the present disclosure (e.g., Figure 4 The diagram below shows the structure of the terminal device or server 400. The terminal device in this embodiment may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and vehicle terminals (e.g., vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 4The electronic device shown is merely an example and should not bring any limitation to the functions and usage range of the embodiments of the present disclosure.

[0131] As shown in Figure 4 The electronic device 400 can include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 401 that can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 402 or programs loaded into a random access memory (RAM) 403 from a storage device 408. In the RAM 403, various programs and data required for the operation of the electronic device 400 are also stored. The processing device 401, the ROM 402, and the RAM 403 are connected to each other through a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0132] Generally, the following devices can be connected to the I / O interface 405: input devices 406 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 407 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 408 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 409. The communication devices 409 can allow the electronic device 400 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 4 The electronic device 400 with various devices is shown, but it should be understood that it is not required to implement or have all the devices shown. More or less devices can alternatively be implemented or included.

[0133] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network through the communication devices 409, or installed from the storage devices 408, or installed from the ROM 402. When the computer program is executed by the processing device 401, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are performed.

[0134] The names of the messages or information exchanged between the plurality of devices in the embodiments of the present disclosure are only for illustrative purposes, and are not intended to limit the scope of the messages or information.

[0135] The electronic device provided by the embodiments of the present disclosure belongs to the same inventive concept as the video processing method provided by the above-mentioned embodiments, and the technical details not described in detail in the present embodiments can be referred to the above-mentioned embodiments, and the present embodiments have the same beneficial effects as the above-mentioned embodiments.

[0136] The computer storage medium provided by the embodiments of the present disclosure stores a computer program, and the computer program is executed by a processor to implement the video processing method provided by the above-mentioned embodiments.

[0137] It should be noted that the computer readable medium of the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or apparatus. In the present disclosure, the computer readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer readable program code. Such a propagated data signal can take various forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. The computer readable signal medium can also be any computer readable medium other than the computer readable storage medium, which can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or apparatus. The program code contained in the computer readable medium can be transmitted by any suitable medium, including but not limited to a wire, a cable, a RF (radio frequency) or the like, or any suitable combination of the above.

[0138] In some embodiments, the client, server can communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet, and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.

[0139] The above-mentioned computer readable medium can be included in the above-mentioned electronic device; or can exist separately without being assembled into the electronic device.

[0140] The computer readable medium described above carries one or more programs, when the one or more programs are executed by the electronic device, the electronic device is caused to:

[0141] The computer readable medium described above carries one or more programs, when the one or more programs are executed by the electronic device, the electronic device is caused to:

[0142] In response to the special effect trigger operation, the video frame to be processed is sequentially captured;

[0143] When the touch on the display interface is detected, the target display position of the touch in the video frame to be processed is determined, and a target control object is added at the target display position;

[0144] When it is detected that the special effect playing condition is met, the display form of the target control object is sequentially determined;

[0145] The target control object is controlled to display in the video frame to be processed according to the corresponding display form.

[0146] Computer program code for carrying out operations of the present disclosure can be written in one or more programming languages or combinations of languages including object oriented programming languages such as Java, Smalltalk, C++ or conventional procedural programming languages such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0147] The computer program product of the first aspect can include one or more non-transitory computer-readable media storing instructions that, when executed, cause one or more processors to perform the operations of the first aspect. The one or more non-transitory computer-readable media can include, for example, magnetic media such as one or more magnetic disks, magnetic tapes or cassettes; optical media such as one or more compact discs, optical discs or Blu-ray discs; magneto-optical media such as one or more floptical discs; solid state media such as one or more solid state drives or other flash memory arrays; or any suitable combination of these. The one or more non-transitory computer-readable media can be encoded with instructions that, when executed, cause one or more processors to perform the operations of the first aspect.

[0148] The units described in the embodiments of the present disclosure can be implemented by software, or by hardware, or by a combination of software and hardware. In some cases, the name of the unit does not constitute a limitation on the unit itself. For example, the first obtaining unit can also be described as a unit for obtaining at least two Internet protocol addresses.

[0149] The functions described in this document can be implemented in part or in whole using one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Program-specific Integrated Circuits (ASICs), Program-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

[0150] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more of: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0151] The above description merely illustrates the preferred embodiments of the disclosure and a principle for applying the technologies. It is understood by those skilled in the art that the disclosed scope of the disclosure is not limited to the technical solutions formed by the specific combinations of the technical features described above, and should also cover other technical solutions formed by the combinations of the technical features described above or their equivalent features without departing from the disclosed concept. For example, the technical solutions formed by the mutual replacement of the above-described features and the technical features with similar functions disclosed in the disclosure (but not limited to) can be formed.

[0152] Further, although operations are depicted in a particular, sequential order, this should not be understood as requiring or implying that the operations are performed in the order illustrated or sequentially. In certain circumstances, multitasking and parallel processing can be advantageous. Likewise, although specific implementation details are included for the purpose of providing a thorough disclosure, these should not be construed as limitations on the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable sub-combination.

[0153] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A method of video processing, the method comprising: The method comprises the following steps: in response to a special effect triggering operation, sequentially acquiring a to-be-processed video frame; when a touch on a display interface is detected, determining a target display position of the touch in the to-be-processed video frame, and adding a target control object at the target display position; when a special effect playing condition is detected, sequentially determining a display form of the target control object; controlling the target control object to display in the to-be-processed video frame according to the corresponding display form; wherein the target display position is a position for displaying the target control object; the sequentially determining the display form of the target control object comprises: determining the display form of the target control object based on a pre-set motion attribute corresponding to the target control object; or determining the display form of the target control object according to a sliding operation on the display interface; the determining the display form of the target control object according to the sliding operation on the display interface comprises: if the target control object is a starting control object, determining motion information and the display form of the target control object based on the sliding operation on the starting control object; determining motion information and a display form of a next target control object adjacent to the target control object according to the motion information and a motion direction of the target control object; wherein the motion information comprises a motion speed, and the display form comprises an upright state or a flat state.

2. The method of claim 1, wherein, Further comprising: based on the obtained spatial position information of the terminal device, determining a target virtual plane located in the to-be-processed video, so as to add the target control object on the target virtual plane based on the touch.

3. The method of claim 1, wherein, the determining the target display position of the touch in the to-be-processed video frame comprises: when a triggering operation on the display interface is detected, determining the touch; based on the touch, determining the target display position.

4. The method of claim 1, wherein, In the process of sequentially acquiring the to-be-processed video frame, further comprising: displaying at least one to-be-selected track; selecting a to-be-selected track triggered last in a first preset time period as a target track, and displaying a guide image corresponding to the target track on the display interface, so that the touch determines a target display position of the target control object based on the guide image; or if none of the at least one to-be-selected track is triggered in the first preset time period, removing the at least one to-be-selected track from the to-be-processed video frame.

5. The method of claim 1, wherein, Further comprising: when it is detected that a triggering operation on the target control object meets a withdrawal condition, removing the target control object from the to-be-processed video frame; wherein the withdrawal condition comprises at least one of the following: the triggering operation is an operation of clicking a withdrawal control, an operation of double-clicking the target control object, and an operation of long-pressing the target control object.

6. The method of claim 1, wherein, The special effect playing condition comprises at least one of the following: a time length of adding the target control object reaches a second preset time length; a control for triggering completion of adding the target control object is triggered.

7. The method of claim 1, wherein, Before the sequentially determining the display form of the target control object, further comprising: determine a preset motion attribute corresponding to the target manipulation object, to determine the display form of the target manipulation object based on the motion attribute; or determine a sliding operation on the display interface, and determine a motion direction of the target manipulation object based on the sliding operation, to determine the display form of the target manipulation object based on the motion direction.

8. The method of claim 7, wherein, Further comprising: determine a target manipulation object corresponding to the sliding operation, and take the target manipulation object as a starting manipulation object for determining the display form of each target manipulation object.

9. The method of claim 1, wherein, The determining of the motion information and the display form of the next target manipulation object adjacent to the target manipulation object based on the motion information and the motion direction of the target manipulation object comprises: determining the motion information and the corresponding display form of the next target manipulation object based on the motion information corresponding to the display form of the target manipulation object, the interval information corresponding to the next target manipulation object, and the quality information of the next target manipulation object.

10. The method of claim 1, wherein, The controlling of the target manipulation object to display in the to-be-processed video frame according to the corresponding display form comprises: displaying the to-be-processed video frame as background information in the process of displaying the target manipulation object according to the corresponding display form.

11. A video processing apparatus, comprising: Comprising: a video frame acquisition module, configured to acquire to-be-processed video frames in sequence in response to a special effect triggering operation; an object adding module, configured to determine a target display position of a touch point on the to-be-processed video frame when detecting the touch point on the display interface, and add a target manipulation object at the target display position; the target display position is a position for displaying the target manipulation object; a display form determining module, configured to determine the display form of the target manipulation object in sequence when detecting that a special effect playing condition is met; a special effect display module, configured to control the target manipulation object to display in the to-be-processed video frame according to the corresponding display form. The display form determining module is specifically configured to determine the display form of the target manipulation object based on a preset motion attribute corresponding to the target manipulation object, or determine the display form of the target manipulation object according to a sliding operation on the display interface. The display form determining module is further configured to, if the target manipulation object is a starting manipulation object, determine the motion information and the display form of the target manipulation object based on a sliding operation on the starting manipulation object, and determine the motion information and the display form of a next target manipulation object adjacent to the target manipulation object based on the motion information and the motion direction of the target manipulation object; the motion information includes a motion speed, and the display form includes an upright state or a flat state.

12. An electronic device, comprising: The electronic device comprises: one or more processors; a storage device configured to store one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the video processing method in any one of claims 1-10.

13. A storage medium containing computer-executable instructions for performing the video processing method of any one of claims 1-10 when executed by a computer processor.

Images